Json Format Lineage Model
Last updated
Last updated
This page gives a detail reference of the data lineage response in Json format. Json lineage model can be returned by SQLFlow server or Dlineage tool with /json
flag. The SQLFlow UI also get this result from the /sqlflow/generation/sqlflow/graph endpoint.
Please refer to for the XML response.
Let's get into details and check the data lineage json resposne:
code: Http Status code, 200 for OK. 4XX **** for cases in which the client seems to have erred such as no authorization or bad request. 500 for internal server error. Error messages would be present in error
If the code is not 200 (request is not a success).
data: data payload
mode: data mode. Could be global
or summary
. Will be set to _ summary
_ mode when the relation number exceeds the relation_limit
global
show all data
summary
only share the statics information and there's no graph information. No field data in the table and only table info. Users need to invoke to get the field data in detail.
: payload for statics information in summary
mode
: data model of the analysis result
: graph model of the analysis result
sessionId: session id, used to get the cache information in
jobId: job id, used to get the cache informaion in
error: contains error messages if the status code is not 200
database: database number
schema: schema number
table: table number
view: view number
column: column number
relationship: relationship number
process: process number
mostRelationTables: the top three tables which contain the most relationships
sqlflow payload contains two nodes. dbojbs and relationship.
The top element of the dbobjs payload is an array and the array representing different server instances. For each server instance, we will have:
dbVendor: database type
name: instance name
databases: present if support database
schemas: present if database is not supported and only schema is supported.
dbLinks: dbLinks will be present if the resposne json is generated from metadata. Will not be present If the response json is generated from dataflow
queries: present if the response json is generated from metadata
if supportsCatalogs=true,supportsSchemas=true:
server-->database-->schema-->tables/views/others/packages/procedures/functions/triggers
if supportsCatalogs=true,supportsSchemas=false:
server-->database-->tables/views/others/packages/procedures/functions/triggers
if supportsCatalogs = false, supportsSchemas = true:
server --> schema --> tables/views/others/packages/procedures/functions/triggers
Database node and Schema node may contain other information indicating the data of procedure
, trigger
, function
Relationship is the atom unit of the data lineage. Relationship builds a link between the source and target column (column-level lineage).
A relation includes the type
, target
, sources
and other attributes.
id: relation id
type: relation type, could be fdd
, fdr
, join
, or call
function: present if the relationship is about function
effectType: effect type of the relation, based on STMT
processId: process id by which the relation is generated
timestampMin: the earliest time when the relationship is generated
timestampMax: the latest time when the relationship is generated
id
name
column
columnType
sourceId
sourceName
parentId
parentName
clauseType
function
type
coordinates
Check here for more details
relationIdMap:
Mapping list between the graph ui id and the relationship id
listIdMap:
Mapping list between graph ui id and the graph model id
elements:
tables
id: table id, will be generated into UI model by mappings in the listIdMap
table: table name
width: table width
height: table height
x:table x-axis (horizontal) coordinate
y:table y-axis (vertical) coordinate
columns
id: columns id, will be generated into UI model by mappings in the listIdMap
x:column x-axis (horizontal) coordinate
y:column y-axis (vertical) coordinate
edges
id: edge id, mapped with relationship id and type through relationIdMap
sourceId: source column id
targetId: target column id
: metadata, contains information of instance, db, schema, table, view, storage procedure, function, trigger, dblink, sequence, ddl etc..
: relationships after analyzing sql
supportsCatalogs: whether support database (check for more details on this flag)
supportsSchemas: whether support schema (check for more details on this flag)
tables, columns, package, prcedure, argument, process: check for more details
tips: the above structure is same as the servers
part of the metadata result from as well as .
There are tree types for the server instance (same logic ):
Check to get a full database list and the type details.
target: relation target, of structure
sources: relation sources, belongs to structure
caller: caller if the type is call
, belongs to structure
callees: callees if the type is call
, is an array of objects
transforms: array of