Clickhouse 101
Clickhouse container
- https://hub.docker.com/r/clickhouse/clickhouse-server
- container exposes port 8123 for the HTTP interface and port 9000 for the native client
- https://dbeaver.com/docs/dbeaver/Clickhouse/ «< client to use with Clickhouse
- volumes:
/var/lib/clickhouse/- main folder where ClickHouse stores the data/var/log/clickhouse-server/- logs/etc/clickhouse-server/config.d/*.xml- files with server configuration adjustmenets/etc/clickhouse-server/users.d/*.xml- files with user settings adjustmenets/docker-entrypoint-initdb.d/- folder with database initialization scripts
❯ docker pull clickhouse/clickhouse-server:25.2.1
❯ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
clickhouse/clickhouse-server 25.2.1 15f846e024cc 2 days ago 676MB
Server
docker run -d --name clh-server \
--ulimit nofile=262144:262144 \
-e CLICKHOUSE_USER=myuser \
-e CLICKHOUSE_PASSWORD=mypwd \
-p 18123:8123 \
clickhouse/clickhouse-server:25.2.1
# alternatively mount these or more...
-v "$PWD/ch_data:/var/lib/clickhouse/" \
-v "$PWD/ch_logs:/var/log/clickhouse-server/"
- By default, ClickHouse will be accessible only via the Docker network. See the networking section below.
- By default, starting above server instance will be run as the default user without password.
- The container exposes ports:
8123for the HTTP interface9000for the native client
Client
- https://clickhouse.com/docs/en/interfaces/cli
- https://clickhouse.com/docs/en/interfaces/http
- https://clickhouse.com/docs/en/interfaces/third-party/client-libraries
- https://clickhouse.com/docs/en/interfaces/third-party/integrations
docker exec -it clh-server clickhouse-client
ClickHouse CLI
ee90a0d2dd2a :) show databases
┌─name───────────────┐
1. │ INFORMATION_SCHEMA │
2. │ default │
3. │ information_schema │
4. │ system │
└────────────────────┘
ee90a0d2dd2a :) use system
ee90a0d2dd2a :) show tables
┌─name───────────────────────────┐
1. │ aggregate_function_combinators │
2. │ asynchronous_inserts │
3. │ asynchronous_loader │
...
0c0f4f4a650d :) DESCRIBE TABLE hackernews
┌─name────────┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐
1. │ id │ Nullable(Int64) │ │ │ │ │ │
2. │ deleted │ Nullable(Int64) │ │ │ │ │ │
...
) CREATE DATABASE electricity;
) CREATE TABLE electricity.consumption;
) SELECT COUNT() FROM electricity.consumption;
) SELECT * FROM electricity.consumption LIMIT 5;
) SELECT * FROM electricity.consumption WHERE x=1;
ClickHouse SQL
- ClickHouse SQL is case insensitive
- Any number of space symbols (space, tab, line feed, CR, and form feed) are allowed in a query, including at the beginning and the end of a query
- SQL statement may have the following elements:
Keywords- consist of words that are part of programming syntaxIdentifiers- are the names of the database, tables, schema, or columns/aliases. cannot be a keyword unless they are enclosed by double quotes or backticksClauses- are part of the SQL statement and they are used to specify the conditions using which the data is retrieved from the tableExpressions- are a combination of operators, values, and functions that evaluate to a Boolean, numeric, or date valueQueries- read and return data from the database based on the criteria specifiedStatements- act upon the data stored in the database tables and can have a persistent effect on the dataComments-
Docs
KnowledgeBase
- https://clickhouse.com/docs/knowledgebase/dbms-naming
- https://clickhouse.com/docs/knowledgebase/production
- https://clickhouse.com/docs/knowledgebase/time-series
- https://clickhouse.com/docs/knowledgebase/key-value
- https://clickhouse.com/docs/knowledgebase/change-the-prompt-in-clickhouse-client
- https://clickhouse.com/docs/knowledgebase/add-column
- https://clickhouse.com/docs/knowledgebase/why_is_my_primary_key_not_used
- https://clickhouse.com/docs/knowledgebase/tips-tricks-optimizing-basic-data-types-in-clickhouse
- https://clickhouse.com/docs/knowledgebase/check-query-cache-in-use
- https://clickhouse.com/docs/knowledgebase/check_query_processing_time_only
- https://clickhouse.com/docs/knowledgebase/find-expensive-queries
- https://clickhouse.com/docs/knowledgebase/finding_expensive_queries_by_memory_usage
- https://clickhouse.com/docs/knowledgebase/calculate_ratio_of_zero_sparse_serialization
- https://clickhouse.com/docs/knowledgebase/python_http_requests
- https://clickhouse.com/docs/knowledgebase/python-clickhouse-connect-example
- https://clickhouse.com/docs/knowledgebase/how-to-filter-a-clickhouse-table-by-an-array-column
- https://clickhouse.com/docs/knowledgebase/using-array-join-to-extract-and-query-attributes
- https://clickhouse.com/docs/knowledgebase/filtered-aggregates
- https://clickhouse.com/docs/knowledgebase/compare_resultsets
- https://clickhouse.com/docs/knowledgebase/mapreduce
- https://clickhouse.com/docs/knowledgebase/maximum_number_of_tables_and_databases
- https://clickhouse.com/docs/knowledgebase/json_extract_example
- https://clickhouse.com/docs/knowledgebase/json_simple_example
- https://clickhouse.com/docs/knowledgebase/json-import
- https://clickhouse.com/docs/knowledgebase/importing-geojason-with-nested-object-array
- https://clickhouse.com/docs/knowledgebase/kafka-clickhouse-json
- https://clickhouse.com/docs/knowledgebase/kafka-to-clickhouse-setup
- https://clickhouse.com/docs/knowledgebase/how-to-insert-all-rows-from-another-table
Links
- https://clickhouse.com/learn
- https://clickhouse.com/docs
- https://clickhouse.com/docs/tutorial
- https://clickhouse.com/videos?category=how-to
- https://clickhouse.com/blog/data-formats-clickhouse-csv-tsv-parquet-native
- https://clickhouse.com/blog/announcing-the-new-sql-playground
- https://clickhouse.com/blog/common-getting-started-issues-with-clickhouse
- https://clickhouse.com/blog/visualizing-data-with-grafana
- https://clickhouse.com/blog/working-with-time-series-data-and-functions-ClickHouse
- https://clickhouse.com/blog/clickhouse-faster-queries-with-projections-and-primary-indexes
- https://clickhouse.com/blog/clickhouse-fully-supports-joins-part1
- https://clickhouse.com/blog/clickhouse-fully-supports-joins-hash-joins-part2
- https://clickhouse.com/blog/clickhouse-fully-supports-joins-full-sort-partial-merge-part3
- https://clickhouse.com/blog/clickhouse-fully-supports-joins-direct-join-part4
- https://clickhouse.com/blog/clickhouse-fully-supports-joins-how-to-choose-the-right-algorithm-part5
- https://clickhouse.com/blog/using-materialized-views-in-clickhouse
- https://clickhouse.com/blog/hash-tables-in-clickhouse-and-zero-cost-abstractions
- https://clickhouse.com/blog/geolocating-ips-in-clickhouse-and-grafana
- https://clickhouse.com/blog/generating-random-test-distribution-data-for-clickhouse
- https://clickhouse.com/blog/monitoring-troubleshooting-insert-queries-clickhouse
- https://clickhouse.com/blog/monitoring-troubleshooting-select-queries-clickhouse
- https://clickhouse.com/blog/storing-log-data-in-clickhouse-fluent-bit-vector-open-telemetry
- https://clickhouse.com/blog/storing-traces-and-spans-open-telemetry-in-clickhouse
- https://clickhouse.com/blog/vector-search-clickhouse-p1
- https://clickhouse.com/blog/vector-search-clickhouse-p2
- https://clickhouse.com/blog/forecasting-using-clickhouse
- https://clickhouse.com/blog/querying-pandas-dataframes
- https://clickhouse.com/blog/sql-dynamic-column-selection-clickhouse
- https://clickhouse.com/blog/monitoring-asynchronous-data-inserts-in-clickhouse
- https://clickhouse.com/blog/supercharge-your-clickhouse-data-loads-part1
- https://clickhouse.com/blog/supercharge-your-clickhouse-data-loads-part2
- https://clickhouse.com/blog/supercharge-your-clickhouse-data-loads-part3
- https://clickhouse.com/blog/clickhouse-architectures-with-docker-compose
- https://clickhouse.com/blog/lz4-compression-in-clickhouse
- https://clickhouse.com/blog/handling-updates-and-deletes-in-clickhouse
- https://clickhouse.com/blog/introduction-to-the-clickhouse-query-cache-and-design
- https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states
- https://clickhouse.com/blog/clickhouse-search-with-inverted-indices
- https://clickhouse.com/blog/using-ttl-to-manage-data-lifecycles-in-clickhouse
- https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse
- https://clickhouse.com/blog/optimize-clickhouse-codecs-compression-schema
- https://clickhouse.com/blog/getting-data-into-clickhouse-part-1
- https://clickhouse.com/blog/getting-data-into-clickhouse-part-2-json
- https://clickhouse.com/blog/getting-data-into-clickhouse-part-3-s3
- https://clickhouse.com/blog/announcing-the-new-sql-playground
- https://clickhouse.com/blog/training-machine-learning-models-with-clickhouse-and-featureform
- https://clickhouse.com/blog/real-world-data-noaa-climate-data