Generation Interface

https://github.com/sqlparser/sqlflow_public/tree/master/api

• Generate sqlflow model

  • /sqlflow/generation/sqlflow

• Export lineage as CSV

  • /sqlflow/generation/sqlflow/exportLineageAsCsv

Generate the data lineage

Send the SQL query and get the data lineage result.

/gspLive_backend/sqlflow/generation/sqlflow

generate sqlflow

post

generate sqlflow model

Query parameters

dbvendorstring · enumRequired

database vendor

Default: dbvoraclePossible values:

simpleOutputbooleanOptional

simple output, ignore the intermediate results, defualt is false.

Default: falseExample: false

ignoreRecordSetbooleanOptional

same as simpleOutput, but will keep output of the top level select list, default is false.

Default: falseExample: false

jobIdstringOptional

give the job Id if need to use the job settings and display the job graph

normalizeIdentifierbooleanOptional

whether normalize the Identifier

Default: true

returnFullRelationshipbooleanOptional

whether return full data relationship. will return only summary info if set to false in case of exceeding the relation limit.

Default: false

showConstantTablebooleanOptional

whether show constant table

Default: false

showRelationTypestringOptional

show relation type, required false, default value is 'fdd', multiple values seperated by comma like fdd,frd,fdr,join. Availables are 'fdd' value of target column from source column, 'frd' the recordset count of target column which is affected by value of source column, 'fdr' value of target column which is affected by the recordset count of source column, 'join' combines rows from two or more tables, based on a related column between them

Default: fdd

showTransformbooleanOptional

whether show transform

Default: false

sqltextstringOptional

sql text from which to generate the sqlflow model

tokenstringRequired

The token is generated from userid and usersecret. It is used in every Api invocation.

treatArgumentsInCountFunctionAsDirectDataflowbooleanOptional

Whether treat the arguments in COUNT function as direct Dataflow

Default: false

userIdstringRequired

the user id of sqlflow web or client

Body

sqlfilestring · binaryOptional

sql file from which to generate the sqlflow model

Responses

200

OK

application/json;charset=utf-8

ResponsestringExample: {"code":200,"data":{"dbvendor":"dbvoracle","dbobjs":[{"id":"8","name":"QUERY INSERT-1","type":"process","coordinates":[{"x":1,"y":1,"hashCode":"0"},{"x":1,"y":73,"hashCode":"0"}],"queryHashId":"04ebc5aec1a07e1db80b0bc798742875"},{"id":"4","name":"RAW_CUSTOMERS","type":"table","columns":[{"id":"5","name":"ID","coordinates":[{"x":1,"y":28,"hashCode":"0"},{"x":1,"y":30,"hashCode":"0"}]},{"id":"6","name":"FIRST_NAME","coordinates":[{"x":1,"y":32,"hashCode":"0"},{"x":1,"y":42,"hashCode":"0"}]},{"id":"7","name":"LAST_NAME","coordinates":[{"x":1,"y":44,"hashCode":"0"},{"x":1,"y":53,"hashCode":"0"}]}],"coordinates":[{"x":1,"y":13,"hashCode":"0"},{"x":1,"y":26,"hashCode":"0"}]}],"relationships":[]},"sessionId":"527f2887da8bd7f5952840aa14cb05f357943aff3f5f02a6f1d7f914bca12971_1664807322126"}

201

Created

401

Unauthorized

403

Forbidden

404

Not Found

post

HTTPcURLJavaScriptPython

POST /api/gspLive_backend/sqlflow/generation/sqlflow HTTP/1.1
Host: api.gudusoft.com
Content-Type: multipart/form-data
Accept: */*
Content-Length: 20

{
  "sqlfile": "binary"
}

{
  "code": 200,
  "data": {
    "dbvendor": "dbvoracle",
    "dbobjs": [
      {
        "id": "8",
        "name": "QUERY INSERT-1",
        "type": "process",
        "coordinates": [
          {
            "x": 1,
            "y": 1,
            "hashCode": "0"
          },
          {
            "x": 1,
            "y": 73,
            "hashCode": "0"
          }
        ],
        "queryHashId": "04ebc5aec1a07e1db80b0bc798742875"
      },
      {
        "id": "4",
        "name": "RAW_CUSTOMERS",
        "type": "table",
        "columns": [
          {
            "id": "5",
            "name": "ID",
            "coordinates": [
              {
                "x": 1,
                "y": 28,
                "hashCode": "0"
              },
              {
                "x": 1,
                "y": 30,
                "hashCode": "0"
              }
            ]
          },
          {
            "id": "6",
            "name": "FIRST_NAME",
            "coordinates": [
              {
                "x": 1,
                "y": 32,
                "hashCode": "0"
              },
              {
                "x": 1,
                "y": 42,
                "hashCode": "0"
              }
            ]
          },
          {
            "id": "7",
            "name": "LAST_NAME",
            "coordinates": [
              {
                "x": 1,
                "y": 44,
                "hashCode": "0"
              },
              {
                "x": 1,
                "y": 53,
                "hashCode": "0"
              }
            ]
          }
        ],
        "coordinates": [
          {
            "x": 1,
            "y": 13,
            "hashCode": "0"
          },
          {
            "x": 1,
            "y": 26,
            "hashCode": "0"
          }
        ]
      }
    ],
    "relationships": []
  },
  "sessionId": "527f2887da8bd7f5952840aa14cb05f357943aff3f5f02a6f1d7f914bca12971_1664807322126"
}

• SQLFlow Cloud Server

curl -X POST "https://api.gudusoft.com/gspLive_backend/sqlflow/generation/sqlflow?showRelationType=fdd" -H  "Request-Origion:testClientDemo" -H  "accept:application/json;charset=utf-8" -H  "Content-Type:multipart/form-data" -F "sqlfile=" -F "dbvendor=dbvoracle" -F "ignoreRecordSet=false" -F "simpleOutput=false" -F "sqltext=CREATE VIEW vsal  as select * from emp" -F "userId=YOUR USER ID HERE"  -F "token=YOUR TOKEN HERE"

• SQLFlow on-premise version

curl -X POST "http://127.0.0.1:8081/gspLive_backend/sqlflow/generation/sqlflow?showRelationType=fdd"    -H  "Request-Origion:testClientDemo" -H  "accept:application/json;charset=utf-8" -H  "Content-Type:multipart/form-data" -F "sqlfile=" -F "dbvendor=dbvoracle" -F "ignoreRecordSet=false" -F "simpleOutput=false" -F "sqltext=CREATE VIEW vsal  as select * from emp" -F "userId=gudu|0123456789" 

Export the data lineage in csv format

Send the SQL file and get the csv result which includes the data lineage.

/gspLive_backend/sqlflow/generation/sqlflow/exportFullLineageAsCsv

export sqlflow full lineage as csv format

post

export sqlflow full lineage as csv format

Authorizations

Query parameters

dbvendorstring · enumRequired

db vendor

Default: dbvoraclePossible values:

delimiterstringOptional

delimiter of the values in CSV, default would be ','

Default: ,

export_include_tablestringOptional

export_include_table

sessionIdstringOptional

sessionId

showConstantTablebooleanOptional

showConstantTable

Default: false

sqltextstringOptional

sql text

tokenstringRequired

The token is generated from userid and usersecret. It is used in every Api invocation.

treatArgumentsInCountFunctionAsDirectDataflowbooleanOptional

Whether treat the arguments in COUNT function as direct Dataflow

Default: false

userIdstringRequired

user id

Body

sqlfilestring · binaryOptional

sqlfile

Responses

200

OK

application/json;charset=utf-8

Responsestring

201

Created

401

Unauthorized

403

Forbidden

404

Not Found

post

HTTPcURLJavaScriptPython

POST /api/gspLive_backend/sqlflow/generation/sqlflow/exportFullLineageAsCsv HTTP/1.1
Host: api.gudusoft.com
Authorization: YOUR_API_KEY
Content-Type: multipart/form-data
Accept: */*
Content-Length: 20

{
  "sqlfile": "binary"
}

text

curl -X POST "https://api.gudusoft.com/gspLive_backend/sqlflow/generation/sqlflow/exportFullLineageAsCsv" -H  "accept:application/json;charset=utf-8" -H  "Content-Type:multipart/form-data" -F "userId=YOUR USER ID HERE" -F "token=YOUR TOKEN HERE" -F "dbvendor=dbvoracle" -F "showRelationType=fdd" -F "sqlfile=@YOUR UPLOAD FILE PATH HERE" --output YOUR DOWNLOAD FILE PATH HERE

Sample:

curl -X POST "https://api.gudusoft.com/gspLive_backend/sqlflow/generation/sqlflow/exportFullLineageAsCsv" -H  "accept:application/json;charset=utf-8" -H  "Content-Type:multipart/form-data" -F "userId=auth0|5fc8e95991a780006f180d4d" -F "token=YOUR TOKEN HERE" -F "dbvendor=dbvoracle" -F "showRelationType=fdd" -F "sqlfile=@c:\prg\tmp\demo.sql" --output c:\prg\tmp\demo.csv

Note:

• -H "Content-Type:multipart/form-data" is required.

• Add @ before the upload file path

• --output is required.

• Optional, if you just want to fetch table to table relations, please add -F "tableToTable=true"

Generate the data lineage image

Send the SQL file and get the image result of the data lineage.

/sqlflow/generation/sqlflow/graph/image

generate sqlflow graph image

post

generate sqlflow graph image

Query parameters

columnLevelstringOptional

columnLevel

dbvendorstring · enumRequired

database vendor

Default: dbvoraclePossible values:

hideColumnbooleanOptional

whether hide column

Default: falseExample: false

ignoreFunctionbooleanOptional

ignore function

Default: falseExample: false

simpleOutputbooleanOptional

simple output, ignore the intermediate results, defualt is false.

Default: falseExample: false

ignoreRecordSetbooleanOptional

same as simpleOutput, but will keep output of the top level select list, default is false.

Default: falseExample: false

jobIdstringOptional

give the job Id if need to use the job settings and display the job graph

normalizeIdentifierbooleanOptional

whether normalize the Identifier

Default: true

showConstantTablebooleanOptional

whether show constant table

Default: false

showLinkOnlybooleanOptional

whether show link only

Default: true

showRelationTypestringOptional

show relation type, required false, default value is 'fdd', multiple values seperated by comma like fdd,frd,fdr,join. Availables are 'fdd' value of target column from source column, 'frd' the recordset count of target column which is affected by value of source column, 'fdr' value of target column which is affected by the recordset count of source column, 'join' combines rows from two or more tables, based on a related column between them

Default: fdd

showTransformbooleanOptional

whether show transform

Default: false

sqltextstringOptional

sql text from which to generate the sqlflow model

tokenstringRequired

The token is generated from userid and usersecret. It is used in every Api invocation.

treatArgumentsInCountFunctionAsDirectDataflowbooleanOptional

Whether treat the arguments in COUNT function as direct Dataflow

Default: false

userIdstringRequired

the user id of sqlflow web or client

Body

sqlfilestring · binaryOptional

sqlfile

Responses

200

OK

image/*

Responsestring · binary

201

Created

401

Unauthorized

403

Forbidden

404

Not Found

post

HTTPcURLJavaScriptPython

POST /api/gspLive_backend/sqlflow/generation/sqlflow/graph/image HTTP/1.1
Host: api.gudusoft.com
Content-Type: multipart/form-data
Accept: */*
Content-Length: 20

{
  "sqlfile": "binary"
}

binary

curl --location --request POST 'https://api.gudusoft.com/gspLive_backend/sqlflow/generation/sqlflow/graph/image' \
    --header 'accept: image/*' \
    --form 'sqlfile=@"/home/test.sql"' \
    --form 'dbvendor="dbvoracle"' \
    --form 'userId="gudu|0123456789"'

Sample result: