system.events
Querying in ClickHouse Cloud
The data in this system table is held locally on each node in ClickHouse Cloud. Obtaining a complete view of all data, therefore, requires the clusterAllReplicas function. See here for further details.
Contains information about the number of events that have occurred in the system. For example, in the table, you can find how many SELECT queries were processed since the ClickHouse server started.
Columns:
event(String) — Event name.value(UInt64) — Number of events occurred.description(String) — Event description.
- OSS
- Cloud
The following events are available in ClickHouse OSS:
| Event | Description |
|---|---|
Query | Number of queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries. |
SelectQuery | Same as Query, but only for SELECT queries. |
InitialQuery | Same as Query, but only counts initial queries (see is_initial_query). |
InitialSelectQuery | Same as InitialQuery, but only for SELECT queries. |
QueriesWithSubqueries | Count queries with all subqueries |
SelectQueriesWithSubqueries | Count SELECT queries with all subqueries |
FileOpen | Number of files opened. |
Seek | Number of times the 'lseek' function was called. |
ReadBufferFromFileDescriptorRead | Number of reads (read/pread) from a file descriptor. Does not include sockets. |
ReadBufferFromFileDescriptorReadBytes | Number of bytes read from file descriptors. If the file is compressed, this will show the compressed data size. |
WriteBufferFromFileDescriptorWrite | Number of writes (write/pwrite) to a file descriptor. Does not include sockets. |
WriteBufferFromFileDescriptorWriteBytes | Number of bytes written to file descriptors. If the file is compressed, this will show compressed data size. |
FileSync | Number of times the F_FULLFSYNC/fsync/fdatasync function was called for files. |
FileSyncElapsedMicroseconds | Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for files. |
IOBufferAllocs | Number of allocations of IO buffers (for ReadBuffer/WriteBuffer). |
IOBufferAllocBytes | Number of bytes allocated for IO buffers (for ReadBuffer/WriteBuffer). |
DiskReadElapsedMicroseconds | Total time spent waiting for read syscall. This include reads from page cache. |
DiskWriteElapsedMicroseconds | Total time spent waiting for write syscall. This include writes to page cache. |
NetworkReceiveElapsedMicroseconds | Total time spent waiting for data to receive or receiving data from network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. |
NetworkSendElapsedMicroseconds | Total time spent waiting for data to send to network or sending data to network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. |
NetworkReceiveBytes | Total number of bytes received from network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. |
NetworkSendBytes | Total number of bytes send to network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. |
GlobalThreadPoolExpansions | Counts the total number of times new threads have been added to the global thread pool. This metric indicates the frequency of expansions in the global thread pool to accommodate increased processing demands. |
GlobalThreadPoolThreadCreationMicroseconds | Total time spent waiting for new threads to start. |
GlobalThreadPoolLockWaitMicroseconds | Total time threads have spent waiting for locks in the global thread pool. |
GlobalThreadPoolJobs | Counts the number of jobs that have been pushed to the global thread pool. |
GlobalThreadPoolJobWaitTimeMicroseconds | Measures the elapsed time from when a job is scheduled in the thread pool to when it is picked up for execution by a worker thread. This metric helps identify delays in job processing, indicating the responsiveness of the thread pool to new tasks. |
LocalThreadPoolExpansions | Counts the total number of times threads have been borrowed from the global thread pool to expand local thread pools. |
LocalThreadPoolShrinks | Counts the total number of times threads have been returned to the global thread pool from local thread pools. |
LocalThreadPoolThreadCreationMicroseconds | Total time local thread pools have spent waiting to borrow a thread from the global pool. |
LocalThreadPoolJobs | Counts the number of jobs that have been pushed to the local thread pools. |
LocalThreadPoolBusyMicroseconds | Total time threads have spent executing the actual work. |
LocalThreadPoolJobWaitTimeMicroseconds | Measures the elapsed time from when a job is scheduled in the thread pool to when it is picked up for execution by a worker thread. This metric helps identify delays in job processing, indicating the responsiveness of the thread pool to new tasks. |
QueryPlanOptimizeMicroseconds | Total time spent executing query plan optimizations. |
ContextLock | Number of times the lock of Context was acquired or tried to acquire. This is global lock. |
RWLockAcquiredReadLocks | Number of times a read lock was acquired (in a heavy RWLock). |
QueryProfilerSignalOverruns | Number of times we drop processing of a query profiler signal due to overrun plus the number of signals that OS has not delivered due to overrun. |
QueryProfilerRuns | Number of times QueryProfiler had been run. |
MainConfigLoads | Number of times the main configuration was reloaded. |
ServerStartupMilliseconds | Time elapsed from starting server to listening to sockets in milliseconds |
AsyncLoaderWaitMicroseconds | Total time a query was waiting for async loader jobs. |
LogTrace | Number of log messages with level Trace |
LogDebug | Number of log messages with level Debug |
LogInfo | Number of log messages with level Info |
LogWarning | Number of log messages with level Warning |
LoggerElapsedNanoseconds | Cumulative time spend in logging |
InterfaceNativeSendBytes | Number of bytes sent through native interfaces |
InterfaceNativeReceiveBytes | Number of bytes received through native interfaces |
MemoryWorkerRun | Number of runs done by MemoryWorker in background |
MemoryWorkerRunElapsedMicroseconds | Total time spent by MemoryWorker for background work |
AsyncLoggingFileLogTotalMessages | How many messages (accepted or dropped) have been sent to the async queue for the file log |
AsyncLoggingErrorFileLogTotalMessages | How many messages (accepted or dropped) have been sent to the async queue for the error file log |
AsyncLoggingTextLogTotalMessages | How many messages (accepted or dropped) have been sent to the async queue for the text_log |
You can find all supported events in source file src/Common/ProfileEvents.cpp.
The following events are available in ClickHouse Cloud:
| Event | Description |
|---|---|
Query | Number of queries to be interpreted and potentially executed. Does not include queries that failed to parse or were rejected due to AST size limits, quota limits or limits on the number of simultaneously running queries. May include internal queries initiated by ClickHouse itself. Does not count subqueries. |
SelectQuery | Same as Query, but only for SELECT queries. |
InitialQuery | Same as Query, but only counts initial queries (see is_initial_query). |
QueriesWithSubqueries | Count queries with all subqueries |
SelectQueriesWithSubqueries | Count SELECT queries with all subqueries |
SelectQueriesWithPrimaryKeyUsage | Count SELECT queries which use the primary key to evaluate the WHERE condition |
QueryTimeMicroseconds | Total time of all queries. |
SelectQueryTimeMicroseconds | Total time of SELECT queries. |
OtherQueryTimeMicroseconds | Total time of queries that are not SELECT or INSERT. |
FileOpen | Number of files opened. |
Seek | Number of times the 'lseek' function was called. |
ReadBufferFromFileDescriptorRead | Number of reads (read/pread) from a file descriptor. Does not include sockets. |
ReadBufferFromFileDescriptorReadBytes | Number of bytes read from file descriptors. If the file is compressed, this will show the compressed data size. |
WriteBufferFromFileDescriptorWrite | Number of writes (write/pwrite) to a file descriptor. Does not include sockets. |
WriteBufferFromFileDescriptorWriteBytes | Number of bytes written to file descriptors. If the file is compressed, this will show compressed data size. |
ReadCompressedBytes | Number of bytes (the number of bytes before decompression) read from compressed sources (files, network). |
CompressedReadBufferBlocks | Number of compressed blocks (the blocks of data that are compressed independent of each other) read from compressed sources (files, network). |
CompressedReadBufferBytes | Number of uncompressed bytes (the number of bytes after decompression) read from compressed sources (files, network). |
OpenedFileCacheHits | Number of times a file has been found in the opened file cache, so we didn't have to open it again. |
OpenedFileCacheMisses | Number of times a file has been found in the opened file cache, so we had to open it again. |
OpenedFileCacheMicroseconds | Amount of time spent executing OpenedFileCache methods. |
IOBufferAllocs | Number of allocations of IO buffers (for ReadBuffer/WriteBuffer). |
IOBufferAllocBytes | Number of bytes allocated for IO buffers (for ReadBuffer/WriteBuffer). |
ArenaAllocChunks | Number of chunks allocated for memory Arena (used for GROUP BY and similar operations) |
ArenaAllocBytes | Number of bytes allocated for memory Arena (used for GROUP BY and similar operations) |
FunctionExecute | Number of SQL ordinary function calls (SQL functions are called on per-block basis, so this number represents the number of blocks). |
TableFunctionExecute | Number of table function calls. |
DefaultImplementationForNullsRows | Number of rows processed by default implementation for nulls in function execution |
DefaultImplementationForNullsRowsWithNulls | Number of rows which contain null values processed by default implementation for nulls in function execution |
MarkCacheHits | Number of times an entry has been found in the mark cache, so we didn't have to load a mark file. |
MarkCacheMisses | Number of times an entry has not been found in the mark cache, so we had to load a mark file in memory, which is a costly operation, adding to query latency. |
QueryConditionCacheHits | Number of times an entry has been found in the query condition cache (and reading of marks can be skipped). Only updated for SELECT queries with SETTING use_query_condition_cache = 1. |
QueryConditionCacheMisses | Number of times an entry has not been found in the query condition cache (and reading of mark cannot be skipped). Only updated for SELECT queries with SETTING use_query_condition_cache = 1. |
CreatedReadBufferOrdinary | Number of times ordinary read buffer was created for reading data (while choosing among other read methods). |
DiskReadElapsedMicroseconds | Total time spent waiting for read syscall. This include reads from page cache. |
DiskWriteElapsedMicroseconds | Total time spent waiting for write syscall. This include writes to page cache. |
NetworkReceiveElapsedMicroseconds | Total time spent waiting for data to receive or receiving data from network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. |
NetworkSendElapsedMicroseconds | Total time spent waiting for data to send to network or sending data to network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. |
NetworkReceiveBytes | Total number of bytes received from network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. |
NetworkSendBytes | Total number of bytes send to network. Only ClickHouse-related network interaction is included, not by 3rd party libraries. |
GlobalThreadPoolExpansions | Counts the total number of times new threads have been added to the global thread pool. This metric indicates the frequency of expansions in the global thread pool to accommodate increased processing demands. |
GlobalThreadPoolThreadCreationMicroseconds | Total time spent waiting for new threads to start. |
GlobalThreadPoolLockWaitMicroseconds | Total time threads have spent waiting for locks in the global thread pool. |
GlobalThreadPoolJobs | Counts the number of jobs that have been pushed to the global thread pool. |
GlobalThreadPoolJobWaitTimeMicroseconds | Measures the elapsed time from when a job is scheduled in the thread pool to when it is picked up for execution by a worker thread. This metric helps identify delays in job processing, indicating the responsiveness of the thread pool to new tasks. |
LocalThreadPoolExpansions | Counts the total number of times threads have been borrowed from the global thread pool to expand local thread pools. |
LocalThreadPoolShrinks | Counts the total number of times threads have been returned to the global thread pool from local thread pools. |
LocalThreadPoolThreadCreationMicroseconds | Total time local thread pools have spent waiting to borrow a thread from the global pool. |
LocalThreadPoolLockWaitMicroseconds | Total time threads have spent waiting for locks in the local thread pools. |
LocalThreadPoolJobs | Counts the number of jobs that have been pushed to the local thread pools. |
LocalThreadPoolBusyMicroseconds | Total time threads have spent executing the actual work. |
LocalThreadPoolJobWaitTimeMicroseconds | Measures the elapsed time from when a job is scheduled in the thread pool to when it is picked up for execution by a worker thread. This metric helps identify delays in job processing, indicating the responsiveness of the thread pool to new tasks. |
RemoteReadThrottlerBytes | Bytes passed through 'max_remote_read_network_bandwidth_for_server'/'max_remote_read_network_bandwidth' throttler. |
RemoteWriteThrottlerBytes | Bytes passed through 'max_remote_write_network_bandwidth_for_server'/'max_remote_write_network_bandwidth' throttler. |
InsertedRows | Number of rows INSERTed to all tables. |
InsertedBytes | Number of bytes (uncompressed; for columns as they stored in memory) INSERTed to all tables. |
ZooKeeperInit | Number of times connection with ZooKeeper has been established. |
ZooKeeperTransactions | Number of ZooKeeper operations, which include both read and write operations as well as multi-transactions. |
ZooKeeperList | Number of 'list' (getChildren) requests to ZooKeeper. |
ZooKeeperCreate | Number of 'create' requests to ZooKeeper. |
ZooKeeperRemove | Number of 'remove' requests to ZooKeeper. |
ZooKeeperExists | Number of 'exists' requests to ZooKeeper. |
ZooKeeperGet | Number of 'get' requests to ZooKeeper. |
ZooKeeperSet | Number of 'set' requests to ZooKeeper. |
ZooKeeperMulti | Number of 'multi' requests to ZooKeeper (compound transactions). |
ZooKeeperMultiRead | Number of read 'multi' requests to ZooKeeper (compound transactions). |
ZooKeeperMultiWrite | Number of write 'multi' requests to ZooKeeper (compound transactions). |
ZooKeeperSync | Number of 'sync' requests to ZooKeeper. These requests are rarely needed or usable. |
ZooKeeperClose | Number of times connection with ZooKeeper has been closed voluntary. |
ZooKeeperWatchResponse | Number of times watch notification has been received from ZooKeeper. |
ZooKeeperWaitMicroseconds | Number of microseconds spent waiting for responses from ZooKeeper after creating a request, summed across all the requesting threads. |
ZooKeeperBytesSent | Number of bytes send over network while communicating with ZooKeeper. |
ZooKeeperBytesReceived | Number of bytes received over network while communicating with ZooKeeper. |
DistributedConnectionTries | Total count of distributed connection attempts. |
DistributedConnectionUsable | Total count of successful distributed connections to a usable server (with required table, but maybe stale). |
SuspendSendingQueryToShard | Total count when sending query to shard was suspended when async_query_sending_for_remote is enabled. |
CompileFunction | Number of times a compilation of generated LLVM code (to create fused function for complex expressions) was initiated. |
CompileExpressionsMicroseconds | Total time spent for compilation of expressions to LLVM code. |
CompileExpressionsBytes | Number of bytes used for expressions compilation. |
QueryPlanOptimizeMicroseconds | Total time spent executing query plan optimizations. |
SelectedParts | Number of data parts selected to read from a MergeTree table. |
SelectedPartsTotal | Number of total data parts before selecting which ones to read from a MergeTree table. |
SelectedRanges | Number of (non-adjacent) ranges in all data parts selected to read from a MergeTree table. |
SelectedMarks | Number of marks (index granules) selected to read from a MergeTree table. |
SelectedMarksTotal | Number of total marks (index granules) before selecting which ones to read from a MergeTree table. |
SelectedRows | Number of rows SELECTed from all tables. |
SelectedBytes | Number of bytes (uncompressed; for columns as they stored in memory) SELECTed from all tables. |
RowsReadByMainReader | Number of rows read from MergeTree tables by the main reader (after PREWHERE step). |
RowsReadByPrewhereReaders | Number of rows read from MergeTree tables (in total) by prewhere readers. |
LoadedDataParts | Number of data parts loaded by MergeTree tables during initialization. |
LoadedDataPartsMicroseconds | Microseconds spent by MergeTree tables for loading data parts during initialization. |
FilteringMarksWithPrimaryKeyMicroseconds | Time spent filtering parts by PK. |
WaitMarksLoadMicroseconds | Time spent loading marks |
BackgroundLoadingMarksTasks | Number of background tasks for loading marks |
MarksTasksFromCache | Number of times marks were loaded synchronously because they were already present in the cache. |
LoadedMarksFiles | Number of mark files loaded. |
LoadedMarksCount | Number of marks loaded (total across columns). |
LoadedMarksMemoryBytes | Size of in-memory representations of loaded marks. |
LoadedPrimaryIndexFiles | Number of primary index files loaded. |
LoadedPrimaryIndexRows | Number of rows of primary key loaded. |
LoadedPrimaryIndexBytes | Number of rows of primary key loaded. |
Merge | Number of launched background merges. |
MergeSourceParts | Number of source parts scheduled for merges. |
MergedRows | Rows read for background merges. This is the number of rows before merge. |
MergedColumns | Number of columns merged during the horizontal stage of merges. |
MergedUncompressedBytes | Uncompressed bytes (for columns as they stored in memory) that was read for background merges. This is the number before merge. |
MergeTotalMilliseconds | Total time spent for background merges |
MergeExecuteMilliseconds | Total busy time spent for execution of background merges |
MergeHorizontalStageTotalMilliseconds | Total time spent for horizontal stage of background merges |
MergeHorizontalStageExecuteMilliseconds | Total busy time spent for execution of horizontal stage of background merges |
MergeVerticalStageTotalMilliseconds | Total time spent for vertical stage of background merges |
MergeProjectionStageTotalMilliseconds | Total time spent for projection stage of background merges |
MergeProjectionStageExecuteMilliseconds | Total busy time spent for execution of projection stage of background merges |
MergePrewarmStageTotalMilliseconds | Total time spent for prewarm stage of background merges |
MergePrewarmStageExecuteMilliseconds | Total busy time spent for execution of prewarm stage of background merges |
MergeTreeDataWriterRows | Number of rows INSERTed to MergeTree tables. |
MergeTreeDataWriterUncompressedBytes | Uncompressed bytes (for columns as they stored in memory) INSERTed to MergeTree tables. |
MergeTreeDataWriterCompressedBytes | Bytes written to filesystem for data INSERTed to MergeTree tables. |
MergeTreeDataWriterBlocks | Number of blocks INSERTed to MergeTree tables. Each block forms a data part of level zero. |
MergeTreeDataWriterBlocksAlreadySorted | Number of blocks INSERTed to MergeTree tables that appeared to be already sorted. |
MergeMutateBackgroundExecutorTaskExecuteStepMicroseconds | Time spent in executeStep() for MergeMutate executor tasks. |
MergeMutateBackgroundExecutorTaskResetMicroseconds | Time spent resetting task for MergeMutate executor. |
CommonBackgroundExecutorTaskExecuteStepMicroseconds | Time spent in executeStep() for Common executor tasks. |
CommonBackgroundExecutorTaskResetMicroseconds | Time spent resetting task for Common executor. |
MergeTreeDataWriterSortingBlocksMicroseconds | Time spent sorting blocks |
InsertedCompactParts | Number of parts inserted in Compact format. |
MergedIntoCompactParts | Number of parts merged into Compact format. |
RegexpWithMultipleNeedlesCreated | Regular expressions with multiple needles (VectorScan library) compiled. |
RegexpWithMultipleNeedlesGlobalCacheHit | Number of times we fetched compiled regular expression with multiple needles (VectorScan library) from the global cache. |
RegexpWithMultipleNeedlesGlobalCacheMiss | Number of times we failed to fetch compiled regular expression with multiple needles (VectorScan library) from the global cache. |
ContextLock | Number of times the lock of Context was acquired or tried to acquire. This is global lock. |
ContextLockWaitMicroseconds | Context lock wait time in microseconds |
RWLockAcquiredReadLocks | Number of times a read lock was acquired (in a heavy RWLock). |
PartsLockHoldMicroseconds | Total time spent holding data parts lock in MergeTree tables |
PartsLockWaitMicroseconds | Total time spent waiting for data parts lock in MergeTree tables |
RealTimeMicroseconds | Total (wall clock) time spent in processing (queries and other tasks) threads (note that this is a sum). |
UserTimeMicroseconds | Total time spent in processing (queries and other tasks) threads executing CPU instructions in user mode. This includes time CPU pipeline was stalled due to main memory access, cache misses, branch mispredictions, hyper-threading, etc. |
SystemTimeMicroseconds | Total time spent in processing (queries and other tasks) threads executing CPU instructions in OS kernel mode. This is time spent in syscalls, excluding waiting time during blocking syscalls. |
MemoryAllocatorPurge | Total number of times memory allocator purge was requested |
MemoryAllocatorPurgeTimeMicroseconds | Total time spent for memory allocator purge |
SoftPageFaults | The number of soft page faults in query execution threads. Soft page fault usually means a miss in the memory allocator cache, which requires a new memory mapping from the OS and subsequent allocation of a page of physical memory. |
OSCPUWaitMicroseconds | Total time a thread was ready for execution but waiting to be scheduled by OS, from the OS point of view. |
OSCPUVirtualTimeMicroseconds | CPU time spent seen by OS. Does not include involuntary waits due to virtualization. |
OSWriteBytes | Number of bytes written to disks or block devices. Doesn't include bytes that are in page cache dirty pages. May not include data that was written by OS asynchronously. |
OSReadChars | Number of bytes read from filesystem, including page cache. |
OSWriteChars | Number of bytes written to filesystem, including page cache. |
QueryProfilerRuns | Number of times QueryProfiler had been run. |
S3ReadMicroseconds | Time of GET and HEAD requests to S3 storage. |
S3ReadRequestsCount | Number of GET and HEAD requests to S3 storage. |
S3ReadRequestsErrors | Number of non-throttling errors in GET and HEAD requests to S3 storage. |
S3ReadRequestAttempts | Number of attempts for GET and HEAD requests, including the initial try and any retries, but excluding retries performed internally by the S3 retry strategy |
S3WriteMicroseconds | Time of POST, DELETE, PUT and PATCH requests to S3 storage. |
S3WriteRequestsCount | Number of POST, DELETE, PUT and PATCH requests to S3 storage. |
S3WriteRequestAttempts | Number of attempts for POST, DELETE, PUT and PATCH requests, including the initial try and any retries, but excluding retries performed internally by the retry strategy |
DiskS3ReadMicroseconds | Time of GET and HEAD requests to DiskS3 storage. |
DiskS3ReadRequestsCount | Number of GET and HEAD requests to DiskS3 storage. |
DiskS3ReadRequestsErrors | Number of non-throttling errors in GET and HEAD requests to DiskS3 storage. |
DiskS3ReadRequestAttempts | Number of attempts for GET and HEAD requests to DiskS3 storage, including the initial try and any retries, but excluding retries performed internally by the S3 retry strategy |
DiskS3WriteMicroseconds | Time of POST, DELETE, PUT and PATCH requests to DiskS3 storage. |
DiskS3WriteRequestsCount | Number of POST, DELETE, PUT and PATCH requests to DiskS3 storage. |
DiskS3WriteRequestAttempts | Number of attempts for POST, DELETE, PUT and PATCH requests to DiskS3 storage, including the initial try and any retries, but excluding retries performed internally by the retry strategy |
S3DeleteObjects | Number of S3 API DeleteObject(s) calls. |
S3ListObjects | Number of S3 API ListObjects calls. |
S3HeadObject | Number of S3 API HeadObject calls. |
S3PutObject | Number of S3 API PutObject calls. |
S3GetObject | Number of S3 API GetObject calls. |
DiskS3DeleteObjects | Number of DiskS3 API DeleteObject(s) calls. |
DiskS3ListObjects | Number of DiskS3 API ListObjects calls. |
DiskS3HeadObject | Number of DiskS3 API HeadObject calls. |
DiskS3PutObject | Number of DiskS3 API PutObject calls. |
DiskS3GetObject | Number of DiskS3 API GetObject calls. |
DiskPlainRewritableS3DirectoryCreated | Number of directories created by the 'plain_rewritable' metadata storage for S3ObjectStorage. |
DiskPlainRewritableS3DirectoryRemoved | Number of directories removed by the 'plain_rewritable' metadata storage for S3ObjectStorage. |
S3Clients | Number of created S3 clients. |
ReadBufferFromS3Microseconds | Time spent on reading from S3. |
ReadBufferFromS3InitMicroseconds | Time spent initializing connection to S3. |
ReadBufferFromS3Bytes | Bytes read from S3. |
WriteBufferFromS3Microseconds | Time spent on writing to S3. |
WriteBufferFromS3Bytes | Bytes written to S3. |
CachedReadBufferReadFromCacheHits | Number of times the read from filesystem cache hit the cache. |
CachedReadBufferReadFromCacheMisses | Number of times the read from filesystem cache miss the cache. |
CachedReadBufferReadFromSourceMicroseconds | Time reading from filesystem cache source (from remote filesystem, etc) |
CachedReadBufferReadFromCacheMicroseconds | Time reading from filesystem cache |
CachedReadBufferReadFromSourceBytes | Bytes read from filesystem cache source (from remote fs, etc) |
CachedReadBufferReadFromCacheBytes | Bytes read from filesystem cache |
CachedReadBufferCacheWriteBytes | Bytes written from source (remote fs, etc) to filesystem cache |
CachedReadBufferCacheWriteMicroseconds | Time spent writing data into filesystem cache |
CachedReadBufferCreateBufferMicroseconds | Prepare buffer time |
CachedWriteBufferCacheWriteBytes | Bytes written from source (remote fs, etc) to filesystem cache |
CachedWriteBufferCacheWriteMicroseconds | Time spent writing data into filesystem cache |
FilesystemCacheLoadMetadataMicroseconds | Time spent loading filesystem cache metadata |
FilesystemCacheCreatedKeyDirectories | Number of created key directories |
FilesystemCacheBackgroundDownloadQueuePush | Number of file segments sent for background download in filesystem cache |
FilesystemCacheLockKeyMicroseconds | Lock cache key time |
FilesystemCacheLockMetadataMicroseconds | Lock filesystem cache metadata time |
FilesystemCacheLockCacheMicroseconds | Lock filesystem cache time |
FilesystemCacheReserveMicroseconds | Filesystem cache space reservation time |
FilesystemCacheReserveAttempts | Filesystem cache space reservation attempt |
FilesystemCacheGetOrSetMicroseconds | Filesystem cache getOrSet() time |
FilesystemCacheGetMicroseconds | Filesystem cache get() time |
FileSegmentCompleteMicroseconds | Duration of FileSegment::complete() in filesystem cache |
FileSegmentLockMicroseconds | Lock file segment time |
FileSegmentWriteMicroseconds | File segment write() time |
FileSegmentUseMicroseconds | File segment use() time |
FileSegmentHolderCompleteMicroseconds | File segments holder complete() time |
FileSegmentFailToIncreasePriority | Number of times the priority was not increased due to a high contention on the cache lock |
FilesystemCacheHoldFileSegments | Filesystem cache file segments count, which were hold |
FilesystemCacheUnusedHoldFileSegments | Filesystem cache file segments count, which were hold, but not used (because of seek or LIMIT n, etc) |
RemoteFSSeeks | Total number of seeks for async buffer |
RemoteFSPrefetches | Number of prefetches made with asynchronous reading from remote filesystem |
RemoteFSCancelledPrefetches | Number of cancelled prefecthes (because of seek) |
RemoteFSUnusedPrefetches | Number of prefetches pending at buffer destruction |
RemoteFSPrefetchedReads | Number of reads from prefecthed buffer |
RemoteFSPrefetchedBytes | Number of bytes from prefecthed buffer |
RemoteFSUnprefetchedReads | Number of reads from unprefetched buffer |
RemoteFSUnprefetchedBytes | Number of bytes from unprefetched buffer |
RemoteFSBuffers | Number of buffers created for asynchronous reading from remote filesystem |
WaitPrefetchTaskMicroseconds | Time spend waiting for prefetched reader |
ThreadpoolReaderTaskMicroseconds | Time spent getting the data in asynchronous reading |
ThreadpoolReaderReadBytes | Bytes read from a threadpool task in asynchronous reading |
ThreadpoolReaderSubmit | Bytes read from a threadpool task in asynchronous reading |
ThreadpoolReaderSubmitReadSynchronously | How many times we haven't scheduled a task on the thread pool and read synchronously instead |
ThreadpoolReaderSubmitReadSynchronouslyBytes | How many bytes were read synchronously |
ThreadpoolReaderSubmitReadSynchronouslyMicroseconds | How much time we spent reading synchronously |
ThreadpoolReaderSubmitLookupInCacheMicroseconds | How much time we spent checking if content is cached |
FileSegmentWaitReadBufferMicroseconds | Metric per file segment. Time spend waiting for internal read buffer (includes cache waiting) |
FileSegmentReadMicroseconds | Metric per file segment. Time spend reading from file |
FileSegmentCacheWriteMicroseconds | Metric per file segment. Time spend writing data to cache |
FileSegmentUsedBytes | Metric per file segment. How many bytes were actually used from current file segment |
ThreadPoolReaderPageCacheMiss | Number of times the read inside ThreadPoolReader was not done from page cache and was hand off to thread pool. |
ThreadPoolReaderPageCacheMissBytes | Number of bytes read inside ThreadPoolReader when read was not done from page cache and was hand off to thread pool. |
ThreadPoolReaderPageCacheMissElapsedMicroseconds | Time spent reading data inside the asynchronous job in ThreadPoolReader - when read was not done from the page cache. |
SynchronousReadWaitMicroseconds | Time spent in waiting for synchronous reads in asynchronous local read. |
AsynchronousRemoteReadWaitMicroseconds | Time spent in waiting for asynchronous remote reads. |
SynchronousRemoteReadWaitMicroseconds | Time spent in waiting for synchronous remote reads. |
MainConfigLoads | Number of times the main configuration was reloaded. |
MetadataFromKeeperCacheHit | Number of times an object storage metadata request was answered from cache without making request to Keeper |
MetadataFromKeeperCacheMiss | Number of times an object storage metadata request had to be answered from Keeper |
MetadataFromKeeperCacheUpdateMicroseconds | Total time spent in updating the cache including waiting for responses from Keeper |
MetadataFromKeeperUpdateCacheOneLevel | Number of times a cache update for one level of directory tree was done |
MetadataFromKeeperTransactionCommit | Number of times metadata transaction commit was attempted |
MetadataFromKeeperCleanupTransactionCommit | Number of times metadata transaction commit for deleted objects cleanup was attempted |
MetadataFromKeeperOperations | Number of times a request was made to Keeper |
MetadataFromKeeperIndividualOperations | Number of paths read or written by single or multi requests to Keeper |
MetadataFromKeeperIndividualOperationsMicroseconds | Time spend during single or multi requests to Keeper |
SharedMergeTreeMetadataCacheHintLoadedFromCache | Number of times metadata cache hint was found without going to Keeper |
ScalarSubqueriesCacheMiss | Number of times a read from a scalar subquery was not cached and had to be calculated completely |
ServerStartupMilliseconds | Time elapsed from starting server to listening to sockets in milliseconds |
MergerMutatorsGetPartsForMergeElapsedMicroseconds | Time spent to take data parts snapshot to build ranges from them. |
MergerMutatorPrepareRangesForMergeElapsedMicroseconds | Time spent to prepare parts ranges which can be merged according to merge predicate. |
MergerMutatorSelectPartsForMergeElapsedMicroseconds | Time spent to select parts from ranges which can be merged. |
MergerMutatorRangesForMergeCount | Amount of candidate ranges for merge |
MergerMutatorPartsInRangesForMergeCount | Amount of candidate parts for merge |
MergerMutatorSelectRangePartsCount | Amount of parts in selected range for merge |
AsyncLoaderWaitMicroseconds | Total time a query was waiting for async loader jobs. |
LogTrace | Number of log messages with level Trace |
LogDebug | Number of log messages with level Debug |
LogInfo | Number of log messages with level Info |
LogWarning | Number of log messages with level Warning |
LogError | Number of log messages with level Error |
LoggerElapsedNanoseconds | Cumulative time spend in logging |
InterfaceHTTPSendBytes | Number of bytes sent through HTTP interfaces |
InterfaceHTTPReceiveBytes | Number of bytes received through HTTP interfaces |
InterfaceNativeSendBytes | Number of bytes sent through native interfaces |
InterfaceNativeReceiveBytes | Number of bytes received through native interfaces |
InterfacePrometheusSendBytes | Number of bytes sent through Prometheus interfaces |
InterfacePrometheusReceiveBytes | Number of bytes received through Prometheus interfaces |
InterfaceInterserverSendBytes | Number of bytes sent through interserver interfaces |
InterfaceInterserverReceiveBytes | Number of bytes received through interserver interfaces |
SharedMergeTreeVirtualPartsUpdates | Virtual parts update count |
SharedMergeTreeVirtualPartsUpdatesByLeader | Virtual parts updates by leader |
SharedMergeTreeVirtualPartsUpdateMicroseconds | Virtual parts update microseconds |
SharedMergeTreeVirtualPartsUpdatesFromZooKeeper | Virtual parts updates count from ZooKeeper |
SharedMergeTreeVirtualPartsUpdatesFromZooKeeperMicroseconds | Virtual parts updates from ZooKeeper microseconds |
SharedMergeTreeVirtualPartsUpdatesPeerNotFound | Virtual updates from peer failed because no one found |
SharedMergeTreeVirtualPartsUpdatesLeaderSuccessfulElection | Virtual parts updates leader election successful |
SharedMergeTreeMergeMutationAssignmentAttempt | How many times we tried to assign merge or mutation |
SharedMergeTreeMergeMutationAssignmentFailedWithNothingToDo | How many times we tried to assign merge or mutation and failed because nothing to merge |
SharedMergeTreePartsKillerRuns | How many times parts killer has been running |
SharedMergeTreePartsKillerMicroseconds | How much time does parts killer main thread takes |
SharedMergeTreeMergeSelectingTaskMicroseconds | Merge selecting task microseconds for SMT |
SharedMergeTreeScheduleDataProcessingJob | How many times scheduleDataProcessingJob called/ |
SharedMergeTreeScheduleDataProcessingJobNothingToScheduled | How many times scheduleDataProcessingJob called but nothing to do |
SharedMergeTreeScheduleDataProcessingJobMicroseconds | scheduleDataProcessingJob execute time |
SharedMergeTreeHandleBlockingPartsMicroseconds | Time of handling blocking parts in scheduleDataProcessingJob |
SharedMergeTreeHandleFetchPartsMicroseconds | Time of handling fetched parts in scheduleDataProcessingJob |
SharedMergeTreeHandleOutdatedPartsMicroseconds | Time of handling outdated parts in scheduleDataProcessingJob |
SharedMergeTreeTryUpdateDiskMetadataCacheForPartMicroseconds | Time of tryUpdateDiskMetadataCacheForPart in scheduleDataProcessingJob |
DiskConnectionsCreated | Number of created connections for disk |
DiskConnectionsReused | Number of reused connections for disk |
DiskConnectionsReset | Number of reset connections for disk |
DiskConnectionsPreserved | Number of preserved connections for disk |
DiskConnectionsExpired | Number of expired connections for disk |
DiskConnectionsElapsedMicroseconds | Total time spend on creating connections for disk |
HTTPConnectionsCreated | Number of created client HTTP connections |
HTTPConnectionsReused | Number of reused client HTTP connections |
HTTPConnectionsPreserved | Number of preserved client HTTP connections |
HTTPConnectionsElapsedMicroseconds | Total time spend on creating client HTTP connections |
HTTPServerConnectionsCreated | Number of created server HTTP connections |
HTTPServerConnectionsReused | Number of reused server HTTP connections |
HTTPServerConnectionsPreserved | Number of preserved server HTTP connections. Connection kept alive successfully |
HTTPServerConnectionsExpired | Number of expired server HTTP connections. |
HTTPServerConnectionsReset | Number of reset server HTTP connections. Server closes connection |
AddressesDiscovered | Total count of new addresses in DNS resolve results for HTTP connections |
ReadWriteBufferFromHTTPRequestsSent | Number of HTTP requests sent by ReadWriteBufferFromHTTP |
ReadWriteBufferFromHTTPBytes | Total size of payload bytes received and sent by ReadWriteBufferFromHTTP. Doesn't include HTTP headers. |
ConcurrencyControlSlotsGranted | Number of CPU slot granted according to guarantee of 1 thread per query and for queries with setting 'use_concurrency_control' = 0 |
ConcurrencyControlSlotsAcquiredNonCompeting | Total number of noncompeting CPU slot acquired |
MemoryWorkerRun | Number of runs done by MemoryWorker in background |
MemoryWorkerRunElapsedMicroseconds | Total time spent by MemoryWorker for background work |
FilterTransformPassedRows | Number of rows that passed the filter in the query |
FilterTransformPassedBytes | Number of bytes that passed the filter in the query |
IndexBinarySearchAlgorithm | Number of times the binary search algorithm is used over the index marks |
IndexGenericExclusionSearchAlgorithm | Number of times the generic exclusion search algorithm is used over the index marks |
AsyncLoggingConsoleTotalMessages | How many messages (accepted or dropped) have been sent to the async queue for the console log |
AsyncLoggingFileLogTotalMessages | How many messages (accepted or dropped) have been sent to the async queue for the file log |
AsyncLoggingErrorFileLogTotalMessages | How many messages (accepted or dropped) have been sent to the async queue for the error file log |
AsyncLoggingTextLogTotalMessages | How many messages (accepted or dropped) have been sent to the async queue for the text_log |
Example
See Also
- system.asynchronous_metrics — Contains periodically calculated metrics.
- system.metrics — Contains instantly calculated metrics.
- system.metric_log — Contains a history of metrics values from tables
system.metricsandsystem.events. - Monitoring — Base concepts of ClickHouse monitoring.