Gudu SQLFlow Product Docs
  • 1. Introduction
    • What is Gudu SQLFlow?
      • What SQLFlow can do
      • Architecture Overview
    • Getting Started
      • Sign up a new account
        • Team Management
        • Delete My Account
        • Activate by entering a coupon
      • How to use SQLFlow
      • Different modes in Gudu SQLFlow
        • Query mode
        • Job mode
      • Basic Usage
      • Convert SQL to E-R Diagram
      • Colors in SQLFlow Diagram
      • Show call relationship
    • Installation
      • Version and Users
        • Cloud and On-Premise version
        • SQLFlow before Version 6
          • For older version SQLFlow under Linux
          • For older version SQLFlow under MacOS
          • For older version SQLFlow under Windows
      • Linux
      • MacOS
      • Windows
      • Docker
      • Clickhouse Installation
        • Clickhouse For CentOs
        • Clickhouse For Ubuntu/Debian/RHEL
      • Troubleshooting
      • Upgrade
      • Third Party Components
      • Renew License File
    • UI
      • SQLText Editor
      • Schema Explorer
      • Diagram Panel
      • Settings
      • Job Management
        • Job Sources
    • Dlineage Tool
      • Overview
      • Usage
        • Analyze data linege from SQL files
        • Analyze data linege from a database
        • Resolve the ambiguous columns in SQL query
        • Map the DataFlowAnalyzer and the settings on SQLFlow UI
        • Settings
      • Dataflow.xml structure
      • FAQ
  • 2. CONCEPTS
    • Data Lineage Basics
      • Dataflow
        • Relations generated by SQLFlow
      • Direct Dataflow
      • Indirect Dataflow
      • Aggregate function and Dataflow
      • Dataflow chain
    • Data Lineage Format Reference
  • 3. API Docs
    • Prerequisites
    • Using the Rest API
    • SQLFlow Rest API reference
      • User Interface
      • Generation Interface
        • /sqlflow
        • /sqlflow/selectedgraph/table_level_lineage
        • /sqlflow/selectedgraph/image
        • /sqlflow/graph
        • /sqlflow/graph/table_level_lineage
        • /sqlflow/graph/image
        • /sqlflow/downstreamGraph
        • /sqlflow/upstreamGraph
        • /sqlflow/erdiagramSelectGraph
        • /sqlflow/leftMostSourceTableGraph
      • Job Interface
        • /submitUserJob
        • /displayUserJobSummary
        • /displayUserJobsSummary
        • /exportLineageAsJson
        • /exportFullLineageAsJson
        • /exportLineageAsGraphml
        • /submitPersistJob
        • /displayUserLatestJobTableLevelLineage
      • Export Image
      • Export CSV
        • /sqlflow/exportFullLineageAsCsv
        • /job/exportFullLineageAsCsv
    • Swagger UI
    • Export the data lineage result
    • Python
      • Basic Usage
      • Advanced Usage
    • SQL Parser API
      • checkSyntax
  • 4. SQLFlow Widget
    • Widget Get started
    • Usages
    • Widget API Reference
  • 5. Databases
    • Database Objects
      • Azure
      • DB2
  • 6. SQLFlow-ingester
    • Introduction
      • SQLFlow-Exporter
      • SQLFlow-Extractor
      • SQLFlow-Submitter
    • Get Started
      • SQL Server
    • SQLFlow-Ingester Java API Usage
    • Understand the format of exported data
      • Oracle
      • Microsoft SQL Server
      • MySQL
      • PostgreSQL
    • List of Supported dbVendors
    • Git Repo
    • Third Party Components
  • 7. Reference
    • Lineage Model
      • Json Format Lineage Model
      • XML Format Lineage Model
      • Data Lineage Elements
    • Database Model
  • 8. other
    • FAQ
      • Handling Internal Database
      • Delete Your Account
      • Table Form Data Without Intermediates
      • Not all schema exported from Oracle
      • Lineage Customization
    • Roadmap
    • SQL Samples
      • Exchange table partition
      • Generate relationship for renamed table
      • Snowflake table function lineage detection
    • Change Logs
    • SQLFlow with Oracle XML functions
    • Major Organizations Utilizing SQLFlow
Powered by GitBook
On this page
  • JDK
  • Clickhouse
  • 1. Download & Install
  • 2. Memory Settings
  • 3. Set Clickhouse default password
  • 4. Init Clickhouse
  • 5. Start SQLFlow
  1. 1. Introduction
  2. Installation
  3. Clickhouse Installation

Clickhouse For Ubuntu/Debian/RHEL

JDK

Using JDK 11 as well as enabling G1 garbage collector is recommended. Comparing to JDK 8, the performance of handling jobs under multi-threads will be improved more than 30%.

Clickhouse

1. Download & Install

sudo apt-get install apt-transport-https ca-certificates dirmngr
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 8919F6BD2B48D754

echo "deb https://packages.clickhouse.com/deb stable main" | sudo tee /etc/apt/sources.list.d/clickhouse.list
sudo apt-get update

sudo apt-get install -y clickhouse-server clickhouse-client
sudo chown -R clickhouse /var/lib/clickhouse /var/log/clickhouse-server /etc/clickhouse-server /etc/clickhouse-client

Please set Clickhouse password to sqlflow@gudu

Generate clickhouse-server key & crts

Check your config.xml

vim /etc/clickhouse-server/config.xml

Find the instruction of openSSL

Run the above command, in my case it is:

sudo openssl req -subj "/CN=localhost" -new -newkey rsa:2048 -days 365 -nodes -x509 -keyout /etc/clickhouse-server/server.key -out /etc/clickhouse-server/server.crt

You should have:

If you got

Can't load /home/<user>/.rnd into RNG

Then you can try removing or commenting RANDFILE = $ENV::HOME/.rnd line in /etc/ssl/openssl.cnf

Replace IPv6 config

If your server is not enabled with IPV6, you will have following errors when trying to start the clickhouse server:

{} <Error> Application: DB::Exception: Listen [::]:8123 failed: Poco::Exception. Code: 1000, e.code() = 0, DNS error: EAI: Address family for hostname not supported (version 22.2.3.1)

This is because the clickhouse server uses IPV6 syntax by default. We need to replace the IPV6 syntax by IPV4 compatible syntax.

Edit the listen.xml:

/etc/clickhouse-server/config.d/listen.xml

Replace the line

<listen_host>::</listen_host>

to

<listen_host>0.0.0.0</listen_host>

Hint: Do not change the same field value in /etc/clickhouse-server/config.xml, if the field is changed in config.xml, please restore the field back to the original value.

2. Memory Settings

We will need to limit the clickhouse service If the client server has a memory less than 32GB.

clickhouse memory settings

In /etc/clickhouse-server/config.xml set max_server_memory_usage_to_ram_ratio to 0.125 so that 32G*0.125 will be 4G

Update /etc/clickhouse-server/users.xml and set max_memory_usage to 4000000000, max_bytes_before_external_group_by to 2000000000. You may need to create max_bytes_before_external_group_by if it isn't present.

Restart the Clickhouse service: sudo /etc/init.d/clickhouse-server restart

Sqlservice memory settings

With this config, the Simple mode will use 22G memory and the Rugular mode will be in 18G memory.

# Update sqlservice.sh, set the heapsize as 18G
elif (( $memory < 32*1024*1024 ));
  then
    heapsize="18g"

3. Set Clickhouse default password

If mistakenly set to another password, you will need:

  1. delete /etc/clickhouse-server/users.d/default-password.xml

  2. restart Clickhouse, sudo /etc/init.d/clickhouse-server restart

4. Init Clickhouse

Run following commands:

cd /wings/sqlflow/backend
sh bin/init_regular.sh

5. Start SQLFlow

cd /wings/sqlflow/backend
sh bin/backend.sh
PreviousClickhouse For CentOsNextTroubleshooting

Last updated 1 year ago

Re- if you meet the following error after properly configuring the listen.xml:

Set Clickhouse password to

update /etc/clickhouse-server/users.xml, set password to

If you got any errors related to Clickhouse, you can check /var/log/clickhouse-sever for the error logs and please ensure that you have finished the steps of and .

sqlflow@gudu
sqlflow@gudu
generate clickhouse-server key & crts
key generation
IPv6 configuration