Run Report

Adapter: kurrentdb   |   Workload: kurrentdb-w16

← Back to workload report

Duration: 15.0s   |   Throughput: 116565 eps   |   Operation Errors: 0

Throughput

Latency

Operation Errors

CPU

Memory

Tool CPU

Tool Memory

Tool Latency

Store Process Resource Metrics

CPU (avg/peak): 648.3% / 694.4%

Memory (avg/peak): 2670 / 3524 MB

Benchmark Process Resource Metrics

CPU (avg/peak): 20.2% / 32.2%

Memory (avg/peak): 114 / 114 MB

Container Stats

Startup Time: 2.68s

Image Size: 335 MB

Container Logs

[    1, 1,07:12:21.249,INF] KurrentDB                      DB VERSION:               25.1.0.2664 ee (3f14c498-aee8-49fc-bb76-b30e2994b4c4/5abd2cabea42a172cc4b6e573f129d02d7420bc2, 2025-10-15 13:05:09.050977)
[    1, 1,07:12:21.256,INF] KurrentDB                      OS ARCHITECTURE:          X64 
[    1, 1,07:12:21.257,INF] KurrentDB                      OS:                       Linux (Unix 7.0.0.29)
[    1, 1,07:12:21.258,INF] KurrentDB                      RUNTIME:                  .NET 8.0.20/574100b69 (64-bit)
[    1, 1,07:12:21.258,INF] KurrentDB                      GC:                       3 GENERATIONS IsServerGC: True Latency Mode: Interactive
[    1, 1,07:12:21.258,INF] KurrentDB                      LOGS:                     /var/log/kurrentdb
[    1, 1,07:12:21.259,INF] KurrentDB                      IsWindowsService:         False
[    1, 1,07:12:21.260,INF] KurrentDB                      GC Configuration settings:
    ServerGC: True
    ConcurrentGC: True
    RetainVM: False
    NoAffinitize: False
    GCCpuGroup: False
    GCLargePages: False
    HeapCount: 8
    MaxHeapCount: 0
    GCHeapAffinitizeMask: 0
    GCHeapAffinitizeRanges: 
    GCHighMemPercent: 0
    GCHeapHardLimit: 19711222579
    GCHeapHardLimitPercent: 60
    GCHeapHardLimitSOH: 0
    GCHeapHardLimitLOH: 0
    GCHeapHardLimitPOH: 0
    GCHeapHardLimitSOHPercent: 0
    GCHeapHardLimitLOHPercent: 0
    GCHeapHardLimitPOHPercent: 0
    GCConserveMem: 0
    GCName: 
    GCDynamicAdaptationMode: 0
[    1, 1,07:12:21.266,INF] KurrentDB                      
MODIFIED OPTIONS:
    APPLICATION OPTIONS:
         INSECURE:                                           true (Environment Variables)
         TELEMETRY OPTOUT:                                   true (Environment Variables)

    CLUSTER OPTIONS:
         CLUSTER SIZE:                                       1 (Environment Variables)

    DATABASE OPTIONS:
         MEM DB:                                             false (Environment Variables)

    INTERFACE OPTIONS:
         ENABLE ATOM PUB OVER HTTP:                          true (Environment Variables)
         NODE IP:                                            0.0.0.0 (Yaml)
         REPLICATION IP:                                     0.0.0.0 (Yaml)

    PROJECTION OPTIONS:
         RUN PROJECTIONS:                                    All (Environment Variables)
         START STANDARD PROJECTIONS:                         true (Environment Variables)


DEFAULT OPTIONS:
    APPLICATION OPTIONS:
         ALLOW ANONYMOUS ENDPOINT ACCESS:                    False ()
         ALLOW ANONYMOUS STREAM ACCESS:                      False ()
         ALLOW UNKNOWN OPTIONS:                              False ()
         CONFIG:                                             /etc/kurrentdb/kurrentdb.conf ()
         DISABLE HTTP CACHING:                               False ()
         ENABLE HISTOGRAMS:                                  False ()
         HELP:                                               False ()
         LOG FAILED AUTHENTICATION ATTEMPTS:                 False ()
         LOG HTTP REQUESTS:                                  False ()
         MAX APPEND EVENT SIZE:                              16777216 ()
         MAX APPEND SIZE:                                    268435456 ()
         OVERRIDE ANONYMOUS ENDPOINT ACCESS FOR GOSSIP:      True ()
         SKIP INDEX SCAN ON READS:                           False ()
         STATS PERIOD SEC:                                   30 ()
         VERSION:                                            False ()
         WHAT IF:                                            False ()
         WORKER THREADS:                                     0 ()

    AUTHENTICATION/AUTHORIZATION OPTIONS:
         AUTHENTICATION CONFIG:                               ()
         AUTHENTICATION TYPE:                                internal ()
         AUTHORIZATION CONFIG:                                ()
         AUTHORIZATION TYPE:                                 internal ()

    CERTIFICATE OPTIONS:
         CERTIFICATE RESERVED NODE COMMON NAME:               ()
         TRUSTED ROOT CERTIFICATES PATH:                     /etc/ssl/certs ()

    CERTIFICATE OPTIONS (FROM FILE):
         CERTIFICATE FILE:                                    ()
         CERTIFICATE PASSWORD:                               ******** ()
         CERTIFICATE PRIVATE KEY FILE:                        ()
         CERTIFICATE PRIVATE KEY PASSWORD:                   ******** ()

    CERTIFICATE OPTIONS (FROM STORE):
         CERTIFICATE STORE LOCATION:                          ()
         CERTIFICATE STORE NAME:                              ()
         CERTIFICATE SUBJECT NAME:                            ()
         CERTIFICATE THUMBPRINT:                              ()
         TRUSTED ROOT CERTIFICATE STORE LOCATION:             ()
         TRUSTED ROOT CERTIFICATE STORE NAME:                 ()
         TRUSTED ROOT CERTIFICATE SUBJECT NAME:               ()
         TRUSTED ROOT CERTIFICATE THUMBPRINT:                 ()

    CLUSTER OPTIONS:
         ARCHIVER:                                           False ()
         CLUSTER DNS:                                        fake.dns ()
         CLUSTER GOSSIP PORT:                                2113 ()
         DEAD MEMBER REMOVAL PERIOD SEC:                     1800 ()
         DISCOVER VIA DNS:                                   True ()
         GOSSIP ALLOWED DIFFERENCE MS:                       60000 ()
         GOSSIP INTERVAL MS:                                 2000 ()
         GOSSIP SEED:                                         ()
         GOSSIP TIMEOUT MS:                                  2500 ()
         LEADER ELECTION TIMEOUT MS:                         1000 ()
         NODE PRIORITY:                                      0 ()
         QUORUM SIZE:                                        1 ()
         READ ONLY REPLICA:                                  False ()
         STREAM INFO CACHE CAPACITY:                         100000 ()
         UNSAFE ALLOW SURPLUS NODES:                         False ()

    DATABASE OPTIONS:
         CACHED CHUNKS:                                      -1 ()
         CHUNK INITIAL READER COUNT:                         5 ()
         CHUNK SIZE:                                         268435456 ()
         CHUNKS CACHE SIZE:                                  536871424 ()
         COMMIT TIMEOUT MS:                                  2000 ()
         DB:                                                 /var/lib/kurrentdb ()
         DB LOG FORMAT:                                      V2 ()
         DISABLE SCAVENGE MERGING:                           False ()
         HASH COLLISION READ LIMIT:                          100 ()
         INDEX:                                               ()
         INDEX CACHE DEPTH:                                  16 ()
         INDEX CACHE SIZE:                                   0 ()
         INITIALIZATION THREADS:                             1 ()
         MAX AUTO MERGE INDEX LEVEL:                         2147483647 ()
         MAX MEM TABLE SIZE:                                 1000000 ()
         MAX TRUNCATION:                                     268435456 ()
         MIN FLUSH DELAY MS:                                 2 ()
         PREPARE TIMEOUT MS:                                 2000 ()
         READER THREADS COUNT:                               0 ()
         REDUCE FILE CACHE PRESSURE:                         False ()
         SCAVENGE BACKEND CACHE SIZE:                        67108864 ()
         SCAVENGE BACKEND PAGE SIZE:                         16384 ()
         SCAVENGE HASH USERS CACHE CAPACITY:                 100000 ()
         SCAVENGE HISTORY MAX AGE:                           30 ()
         SKIP DB VERIFY:                                     False ()
         SKIP INDEX VERIFY:                                  False ()
         STATS STORAGE:                                      File ()
         STREAM EXISTENCE FILTER SIZE:                       256000000 ()
         TRANSFORM:                                          identity ()
         UNBUFFERED:                                         False ()
         UNSAFE DISABLE FLUSH TO DISK:                       False ()
         UNSAFE IGNORE HARD DELETE:                          False ()
         USE INDEX BLOOM FILTERS:                            True ()
         WRITE STATS TO DB:                                  False ()
         WRITE THROUGH:                                      False ()
         WRITE TIMEOUT MS:                                   2000 ()

    DEFAULT USER OPTIONS:
         DEFAULT ADMIN PASSWORD:                             ******** ()
         DEFAULT OPS PASSWORD:                               ******** ()

    DEV MODE OPTIONS:
         DEV:                                                False ()
         REMOVE DEV CERTS:                                   False ()

    GRPC OPTIONS:
         KEEP ALIVE INTERVAL:                                10000 ()
         KEEP ALIVE TIMEOUT:                                 10000 ()

    INTERFACE OPTIONS:
         ADVERTISE HOST TO CLIENT AS:                         ()
         ADVERTISE NODE PORT TO CLIENT AS:                   0 ()
         CONNECTION PENDING SEND BYTES THRESHOLD:            10485760 ()
         CONNECTION QUEUE SIZE THRESHOLD:                    50000 ()
         DISABLE ADMIN UI:                                   False ()
         DISABLE GOSSIP ON HTTP:                             False ()
         DISABLE STATS ON HTTP:                              False ()
         ENABLE TRUSTED AUTH:                                False ()
         ENABLE UNIX SOCKET:                                 False ()
         NODE HOST ADVERTISE AS:                              ()
         NODE PORT:                                          2113 ()
         NODE PORT ADVERTISE AS:                             0 ()
         REPLICATION HEARTBEAT INTERVAL:                     700 ()
         REPLICATION HEARTBEAT TIMEOUT:                      700 ()
         REPLICATION HOST ADVERTISE AS:                       ()
         REPLICATION PORT:                                   1112 ()
         REPLICATION TCP PORT ADVERTISE AS:                  0 ()

    LOGGING OPTIONS:
         DISABLE LOG FILE:                                   False ()
         LOG:                                                /var/log/kurrentdb ()
         LOG CONFIG:                                         logconfig.json ()
         LOG CONSOLE FORMAT:                                 Plain ()
         LOG FILE INTERVAL:                                  Day ()
         LOG FILE RETENTION COUNT:                           31 ()
         LOG FILE SIZE:                                      1073741824 ()
         LOG LEVEL:                                          Default ()

    PROJECTION OPTIONS:
         FAULT OUT OF ORDER PROJECTIONS:                     False ()
         MAX PROJECTION STATE SIZE:                          16777216 ()
         PROJECTION COMPILATION TIMEOUT:                     500 ()
         PROJECTION EXECUTION TIMEOUT:                       250 ()
         PROJECTION THREADS:                                 3 ()
         PROJECTIONS QUERY EXPIRY:                           5 ()

[    1, 1,07:12:21.269,WRN] KurrentDB                      DEPRECATED
AtomPub over HTTP Interface has been deprecated as of version 20.6.0. It is recommended to use gRPC instead

[    1, 1,07:12:21.271,WRN] KurrentDB                      
==============================================================================================================
INSECURE MODE IS ON. THIS MODE IS *NOT* RECOMMENDED FOR PRODUCTION USE.
INSECURE MODE WILL DISABLE ALL AUTHENTICATION, AUTHORIZATION AND TRANSPORT SECURITY FOR ALL CLIENTS AND NODES.
==============================================================================================================

[    1, 1,07:12:21.283,INF] ClusterVNodeHostedService      Cannot find plugins path: /usr/share/kurrentdb/plugins
[    1, 1,07:12:21.316,INF] LogsEndpointPlugin             LogsEndpointPlugin is loaded
[    1, 1,07:12:21.326,INF] ClusterVNodeHostedService      Loaded SubsystemsPlugin plugin: otlp-exporter 25.1.0.2664.
[    1, 1,07:12:21.329,INF] ClusterVNodeHostedService      Loaded SubsystemsPlugin plugin: user-certificates 25.1.0.2664.
[    1, 1,07:12:21.329,INF] ClusterVNodeHostedService      Loaded SubsystemsPlugin plugin: logs-endpoint 25.1.0.2664.
[    1, 1,07:12:21.330,INF] ClusterVNodeHostedService      Loaded SubsystemsPlugin plugin: encryption-at-rest 25.1.0.2664.
[    1, 1,07:12:21.330,INF] ClusterVNodeHostedService      Loaded SubsystemsPlugin plugin: ConnectedSubsystems 0.0.5.
[    1, 1,07:12:21.330,INF] ClusterVNodeHostedService      Loaded SubsystemsPlugin plugin: AutoScavenge 25.1.0.2664.
[    1, 1,07:12:21.331,INF] ClusterVNodeHostedService      Loaded SubsystemsPlugin plugin: tcp-api 25.1.0.2664.
[    1, 1,07:12:21.331,INF] ClusterVNodeHostedService      Loaded SubsystemsPlugin plugin: connectors 25.1.0.2664.
[    1, 1,07:12:21.331,INF] ClusterVNodeHostedService      Loaded SubsystemsPlugin plugin: schema-registry 25.1.0.2664.
[    1, 1,07:12:21.331,INF] ClusterVNodeHostedService      Loaded SubsystemsPlugin plugin: APIV2 25.1.0.2664.
[    1, 1,07:12:21.338,INF] ClusterVNodeHostedService      Using NetMD5 FileHashProvider.
[    1, 1,07:12:21.561,INF] OptionsFormatter               
Experimental Configuration: 
{
  "AsyncIO": false
}
[    1, 1,07:12:21.566,INF] OptionsFormatter               
Archive Configuration: 
{
  "Enabled": false,
  "StorageType": "Unspecified",
  "FileSystem": {
    "Path": ""
  },
  "S3": {
    "Bucket": "",
    "Region": ""
  },
  "RetainAtLeast": {
    "Days": 10675199,
    "LogicalBytes": 9223372036854775807
  }
}
[    1, 1,07:12:21.567,INF] ClusterVNode                   Quorum size set to 1.
[    1, 1,07:12:21.568,INF] ClusterVNode                   Index Path set to /var/lib/kurrentdb/index
[    1, 1,07:12:21.569,DBG] ClusterVNode                   Using File Checkpoints
[    1, 1,07:12:21.575,INF] KurrentDB                      ReaderThreadsCount set to 4. Calculated based on containerized environment and configured value of 0
[    1, 1,07:12:21.575,INF] KurrentDB                      WorkerThreads set to 5. Calculated based on containerized environment and a configured value of 0
[    1, 1,07:12:21.615,INF] OptionsFormatter               
Metrics Configuration: 
{
  "LegacyCoreNaming": false,
  "LegacyProjectionsNaming": false,
  "CoreMeterName": "KurrentDB.Core",
  "ProjectionsMeterName": "KurrentDB.Projections.Core",
  "ServiceName": "kurrentdb",
  "Meters": [
    "KurrentDB.Core",
    "KurrentDB.Projections.Core",
    "Kurrent",
    "Kurrent.Connectors",
    "Kurrent.Connectors.Sinks",
    "KurrentDB.SecondaryIndexes"
  ],
  "Statuses": {
    "Index": true,
    "Node": true,
    "Scavenge": true
  },
  "Checkpoints": {
    "Index": true,
    "Replication": true,
    "Writer": true
  },
  "IncomingGrpcCalls": {
    "Current": true,
    "DeadlineExceeded": true,
    "Failed": true,
    "Total": true,
    "Unimplemented": true
  },
  "GrpcMethods": {
    "StreamAppend": "append",
    "StreamBatchAppend": "append",
    "StreamDelete": "",
    "StreamRead": "read",
    "StreamTombstone": ""
  },
  "Gossip": {
    "ProcessingPushFromPeer": true,
    "ProcessingRequestFromGrpcClient": false,
    "ProcessingRequestFromHttpClient": false,
    "ProcessingRequestFromPeer": false,
    "PullFromPeer": false,
    "PushToPeer": true
  },
  "Kestrel": {
    "ConnectionCount": true
  },
  "System": {
    "Cpu": true,
    "DriveTotalBytes": true,
    "DriveUsedBytes": true,
    "FreeMem": true,
    "LoadAverage15m": true,
    "LoadAverage1m": true,
    "LoadAverage5m": true,
    "TotalMem": true
  },
  "Process": {
    "Cpu": true,
    "DiskReadBytes": true,
    "DiskReadOps": true,
    "DiskWrittenBytes": true,
    "DiskWrittenOps": true,
    "ExceptionCount": true,
    "GcPauseDuration": true,
    "Gen0CollectionCount": true,
    "Gen0Size": true,
    "Gen1CollectionCount": true,
    "Gen1Size": true,
    "Gen2CollectionCount": true,
    "Gen2Size": true,
    "HeapFragmentation": true,
    "HeapSize": true,
    "LockContentionCount": true,
    "LohSize": true,
    "MemPagedBytes": false,
    "MemVirtualBytes": false,
    "MemWorkingSet": true,
    "ThreadCount": true,
    "ThreadPoolPendingWorkItemCount": true,
    "TimeInGc": true,
    "TotalAllocatedBytes": true,
    "UpTime": true
  },
  "Writer": {
    "FlushDuration": true,
    "FlushSize": true
  },
  "Events": {
    "Read": true,
    "Written": true
  },
  "CacheHitsMisses": {
    "Chunk": false,
    "StreamInfo": true
  },
  "ProjectionStats": true,
  "ProjectionExecution": true,
  "ProjectionExecutionByFunction": false,
  "PersistentSubscriptionStats": true,
  "ElectionsCount": true,
  "CacheResources": true,
  "ExpectedScrapeIntervalSeconds": 15,
  "Queues": {
    "Busy": true,
    "Length": true,
    "Processing": false
  },
  "QueueLabels": [
    {
      "Regex": "MainQueue",
      "Label": "Main"
    },
    {
      "Regex": "MonitoringQueue",
      "Label": "Monitoring"
    },
    {
      "Regex": "Timer",
      "Label": "Timer"
    },
    {
      "Regex": "StorageReaderQueue #.*",
      "Label": "Readers"
    },
    {
      "Regex": "StorageWriterQueue",
      "Label": "Writer"
    },
    {
      "Regex": "Subscriptions",
      "Label": "Subscriptions"
    },
    {
      "Regex": "PersistentSubscriptions",
      "Label": "Persistent Subscriptions"
    },
    {
      "Regex": "Projections Leader",
      "Label": "Projections Leader"
    },
    {
      "Regex": "Projection Core #.*",
      "Label": "Projections Core"
    },
    {
      "Regex": "Worker #.*",
      "Label": "Workers"
    },
    {
      "Regex": ".*",
      "Label": "Others"
    }
  ],
  "MessageTypes": [
    {
      "Regex": "CoreMessage-Authentication-.*",
      "Label": "Authentication"
    },
    {
      "Regex": "CoreMessage-Client-.*Completed",
      "Label": "ClientMessageCompleted"
    },
    {
      "Regex": "CoreMessage-Client-ReadEvent",
      "Label": "ReadEvent"
    },
    {
      "Regex": "CoreMessage-Client-.*ReadAllEvents.*",
      "Label": "ReadAll"
    },
    {
      "Regex": "CoreMessage-Client-.*ReadStreamEvents.*",
      "Label": "ReadStream"
    },
    {
      "Regex": "CoreMessage-Client-.*",
      "Label": "OtherClientMessage"
    },
    {
      "Regex": "CoreMessage-Election-.*",
      "Label": "Election"
    },
    {
      "Regex": "CoreMessage-Gossip-.*",
      "Label": "Gossip"
    },
    {
      "Regex": "CoreMessage-Grpc-.*",
      "Label": "Grpc"
    },
    {
      "Regex": "CoreMessage-Http-.*",
      "Label": "Http"
    },
    {
      "Regex": "CoreMessage-Replication-.*",
      "Label": "Replication"
    },
    {
      "Regex": "CoreMessage-ReplicationTracking-.*",
      "Label": "Replication"
    },
    {
      "Regex": "CoreMessage-Storage-.*",
      "Label": "Storage"
    },
    {
      "Regex": "CoreMessage-Subscription-.*",
      "Label": "Subscription"
    },
    {
      "Regex": "CoreMessage-System-.*",
      "Label": "System"
    },
    {
      "Regex": "CoreMessage-Tcp-.*",
      "Label": "Tcp"
    },
    {
      "Regex": "ProjectionMessage-.*",
      "Label": "Projections"
    },
    {
      "Regex": ".*",
      "Label": "Other"
    }
  ]
}
[    1, 1,07:12:21.642,INF] MessageLabelConfigurator       Metrics created 17 message type labels
[    1, 1,07:12:21.681,INF] ClusterVNode                   INSTANCE ID:              e5d74f14-8f6a-4814-a8bf-801a02b65430
[    1, 1,07:12:21.681,INF] ClusterVNode                   DATABASE:                 /var/lib/kurrentdb
[    1, 1,07:12:21.681,INF] ClusterVNode                   WRITER CHECKPOINT:        0 (0x0)
[    1, 1,07:12:21.681,INF] ClusterVNode                   CHASER CHECKPOINT:        0 (0x0)
[    1, 1,07:12:21.681,INF] ClusterVNode                   EPOCH CHECKPOINT:         -1 (0xFFFFFFFFFFFFFFFF)
[    1, 1,07:12:21.681,INF] ClusterVNode                   TRUNCATE CHECKPOINT:      -1 (0xFFFFFFFFFFFFFFFF)
[    1, 1,07:12:21.681,INF] ClusterVNode                   STREAM EXISTENCE FILTER CHECKPOINT: -1 (0xFFFFFFFFFFFFFFFF)
[    1, 1,07:12:21.688,INF] QueueTrackers                  Metrics matched queue Worker #1 with pattern Worker #.*. Label: Workers
[    1, 1,07:12:21.689,INF] QueueTrackers                  Metrics matched queue Worker #2 with pattern Worker #.*. Label: Workers
[    1, 1,07:12:21.689,INF] QueueTrackers                  Metrics matched queue Worker #3 with pattern Worker #.*. Label: Workers
[    1, 1,07:12:21.689,INF] QueueTrackers                  Metrics matched queue Worker #4 with pattern Worker #.*. Label: Workers
[    1, 1,07:12:21.689,INF] QueueTrackers                  Metrics matched queue Worker #5 with pattern Worker #.*. Label: Workers
[    1, 1,07:12:21.724,INF] QueueTrackers                  Metrics matched queue MainQueue with pattern MainQueue. Label: Main
[    1, 1,07:12:21.729,INF] QueueTrackers                  Metrics matched queue MonitoringQueue with pattern MonitoringQueue. Label: Monitoring
[    1, 1,07:12:21.731,INF] StreamExistenceFilter          streamExistenceFilter: Creating /var/lib/kurrentdb/index/stream-existence/streamExistenceFilter.dat
[    1, 1,07:12:21.732,DBG] FileStreamPersistence          Zeroing bloom filter...
[    1, 1,07:12:21.857,DBG] FileStreamPersistence          Done zeroing bloom filter
[    1, 1,07:12:21.857,INF] FileStreamPersistence          Writing header and expanding file...
[    1, 1,07:12:21.859,INF] FileStreamPersistence          Wrote header and expanded file
[    1, 1,07:12:21.859,INF] StreamExistenceFilter          streamExistenceFilter has successfully loaded.
[    1, 1,07:12:21.859,DBG] StreamExistenceFilter          Optimal number of items for a streamExistenceFilter with a configured size of 256 MB is approximately equal to: 251,240,490 with false positive probability: 0.02
[    1, 1,07:12:21.875,INF] QueueTrackers                  Metrics matched queue StorageWriterQueue with pattern StorageWriterQueue. Label: Writer
[    1, 1,07:12:21.881,INF] QueueTrackers                  Metrics matched queue StorageReaderQueue #1 with pattern StorageReaderQueue #.*. Label: Readers
[    1, 1,07:12:21.881,INF] QueueTrackers                  Metrics matched queue StorageReaderQueue #2 with pattern StorageReaderQueue #.*. Label: Readers
[    1, 1,07:12:21.881,INF] QueueTrackers                  Metrics matched queue StorageReaderQueue #3 with pattern StorageReaderQueue #.*. Label: Readers
[    1, 1,07:12:21.881,INF] QueueTrackers                  Metrics matched queue StorageReaderQueue #4 with pattern StorageReaderQueue #.*. Label: Readers
[    1, 1,07:12:21.881,INF] InaugurationManager            Using InaugurationManager
[    1, 1,07:12:21.904,INF] QueueTrackers                  Metrics matched queue Subscriptions with pattern Subscriptions. Label: Subscriptions
[    1, 1,07:12:21.905,INF] QueueTrackers                  Metrics matched queue PersistentSubscriptions with pattern PersistentSubscriptions. Label: Persistent Subscriptions
[    1, 1,07:12:21.910,INF] QueueTrackers                  Metrics matched queue Redaction with pattern .*. Label: Others
[    1, 1,07:12:21.911,INF] QueueTrackers                  Metrics matched queue Timer with pattern Timer. Label: Timer
[    1, 1,07:12:21.912,INF] TelemetrySink                  
Telemetry
---------
You have opted out of sending telemetry by setting the KURRENTDB_TELEMETRY_OPTOUT environment variable to true.
For more information visit https://kurrent.io/telemetry

[    1, 1,07:12:21.927,INF] MiniWeb                        Starting MiniWeb for /web ==> /opt/kurrentdb/clusternode-web
[    1, 1,07:12:22.093,INF] KurrentDB                      Environment Name: Production
[    1, 1,07:12:22.093,INF] KurrentDB                      ContentRoot Path: /opt/kurrentdb/
[    1, 1,07:12:22.132,INF] KurrentDB                      Loaded the following transforms: Identity
[    1, 1,07:12:22.134,INF] KurrentDB                      Active transform set to: Identity
[    1, 1,07:12:22.149,WRN] FileSystemXmlRepository        Storing keys in a directory '/opt/kurrentdb/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed. For more information go to https://aka.ms/aspnet/dataprotectionwarning
[    1, 1,07:12:22.153,INF] OtlpExporterPlugin             OtlpExporter 25.1.0.2664 plugin disabled. No KurrentDB:OpenTelemetry:Otlp configuration found. Not exporting metrics and logs.
[    1, 1,07:12:22.153,INF] UserCertificatesPlugin         UserCertificates 25.1.0.2664 plugin disabled. Set 'KurrentDB:UserCertificates:Enabled' to 'true' to enable user X.509 certificate authentication
[    1, 1,07:12:22.215,INF] LogsEndpointPlugin             LogsEndpoint: Failed to get license, endpoint will not be available.
[    1, 1,07:12:22.215,INF] LogsEndpointPlugin             LogsEndpoint: Serving logs from /var/log/kurrentdb/0.0.0.0-2113-cluster-node at endpoint /admin/logs
[    1, 1,07:12:22.217,INF] EncryptionAtRestPlugin         EncryptionAtRest 25.1.0.2664 plugin disabled. Set 'KurrentDB:EncryptionAtRest:Enabled' to 'true' to enable the encryption at rest plugin
[    1, 1,07:12:22.217,INF] ConnectedSubsystemsPlugin      ConnectedSubsystems 0.0.5 plugin enabled.
[    1, 1,07:12:22.221,INF] AutoScavengePlugin             AutoScavenge is not licensed, stopping.
[    1, 1,07:12:22.225,INF] AutoScavengePlugin             AutoScavenge 25.1.0.2664 plugin disabled. AutoScavenge is not licensed, stopping.
[    1, 1,07:12:22.225,INF] TcpApiPlugin                   TcpApi 25.1.0.2664 plugin disabled. Set 'KurrentDB:TcpPlugin:EnableExternalTcp' to 'true' to enable
[    1, 1,07:12:22.333,INF] ConnectorsPlugin               Connectors 25.1.0.2664 plugin enabled.
[    1, 1,07:12:22.352,INF] SchemaRegistryPlugin           SchemaRegistry 25.1.0.2664 plugin enabled.
[    1, 1,07:12:22.360,INF] ApiV2Plugin                    APIV2 25.1.0.2664 plugin enabled.
[    1, 1,07:12:22.360,INF] NetMD5Provider                 NetMD5 25.1.0.2664 plugin enabled.
[    1, 1,07:12:22.361,INF] QueueTrackers                  Metrics matched queue Projections Leader with pattern Projections Leader. Label: Projections Leader
[    1, 1,07:12:22.361,INF] QueueTrackers                  Metrics matched queue Projections Leader with pattern Projections Leader. Label: Projections Leader
[    1, 1,07:12:22.364,INF] QueueTrackers                  Metrics matched queue Projection Core #0 with pattern Projection Core #.*. Label: Projections Core
[    1, 1,07:12:22.365,INF] QueueTrackers                  Metrics matched queue Projection Core #0 output with pattern Projection Core #.*. Label: Projections Core
[    1, 1,07:12:22.368,INF] QueueTrackers                  Metrics matched queue Projection Core #1 with pattern Projection Core #.*. Label: Projections Core
[    1, 1,07:12:22.368,INF] QueueTrackers                  Metrics matched queue Projection Core #1 output with pattern Projection Core #.*. Label: Projections Core
[    1, 1,07:12:22.369,INF] QueueTrackers                  Metrics matched queue Projection Core #2 with pattern Projection Core #.*. Label: Projections Core
[    1, 1,07:12:22.370,INF] QueueTrackers                  Metrics matched queue Projection Core #2 output with pattern Projection Core #.*. Label: Projections Core
[    1, 1,07:12:22.373,INF] MiniWeb                        Starting MiniWeb for /web/es/js/projections ==> /opt/kurrentdb/projections
[    1, 1,07:12:22.373,INF] MiniWeb                        Starting MiniWeb for /web/es/js/projections/v8/Prelude ==> /opt/kurrentdb/Prelude
[    1, 1,07:12:22.400,INF] DuckDBConnectionPoolLifetime   Created DuckDB connection pool at /var/lib/kurrentdb/kurrent.ddb
[    1, 1,07:12:22.460,INF] DefaultIndexProcessor          Last known log position: C:-1/P:-1
[    1, 1,07:12:22.461,INF] SecondaryIndexingPlugin        SecondaryIndexes 25.1.0.2664 plugin enabled.
[    1, 1,07:12:22.461,INF] PassthroughAuthorizationProvider PassthroughAuthorization 25.1.0.2664 plugin enabled.
[    1, 1,07:12:22.461,INF] PassthroughAuthenticationProvider insecure 25.1.0.2664 plugin enabled.
[    1, 1,07:12:22.461,INF] DelegatedAuthenticationProvider insecure 25.1.0.2664 plugin enabled.
[    1, 1,07:12:22.462,INF] LicensingPlugin                Licensing 25.1.0.2664 plugin enabled.
[    1,11,07:12:22.528,INF] GcSuspensionMetric             Start of full blocking garbage collection at 08/09/2026 07:12:22. GC: #1. Generation: 2. Reason: LargeObjectHeapAllocation. Type: BlockingOutsideBackgroundGC.
[    1,11,07:12:22.529,INF] GcSuspensionMetric             End of full blocking garbage collection at 08/09/2026 07:12:22. GC: #1. Took: 14ms
[    1, 1,07:12:22.639,INF] ConnectorsLicenseService       Allowed Connectors: ["SerilogSink","HttpSink"]
[    1, 1,07:12:22.660,WRN] XmlKeyManager                  No XML encryptor configured. Key {a4930575-de18-4298-92ec-e607de1f3ea3} may be persisted to storage in unencrypted form.
[    1, 1,07:12:22.670,DBG] SchemaRegistryStartupTask      Data Protection Schema Registration completed
[    1, 1,07:12:22.672,DBG] SchemaRegistryStartupTask      Connectors Control Schema Registration completed
[    1, 9,07:12:22.674,INF] ShutdownService                ========== [0.0.0.0:2113] Component 'ConnectorsController' is registered for graceful termination
[    1, 1,07:12:22.676,DBG] SchemaRegistryStartupTask      Connectors Management Schema Registration completed
[    1, 6,07:12:22.676,INF] ShutdownService                ========== [0.0.0.0:2113] Component 'ConnectorsStateProjection' is registered for graceful termination
[    1, 1,07:12:22.677,INF] SystemStartupManager           System startup tasks started
[    1, 1,07:12:22.684,DBG] SchemaMessageRegistrationStartupTask Schema Registry Message Registration completed
[    1, 8,07:12:22.685,INF] ShutdownService                ========== [0.0.0.0:2113] Component 'DuckDBProjector' is registered for graceful termination
[    1, 9,07:12:22.701,INF] TFChunkDb                      Ensuring no excessive chunks...
[    1, 9,07:12:22.703,INF] TFChunkDb                      Done ensuring no excessive chunks.
[    1, 9,07:12:22.703,INF] TFChunkDb                      Removing old chunk versions...
[    1, 9,07:12:22.704,INF] TFChunkDb                      Done removing old chunk versions.
[    1, 9,07:12:22.704,INF] TFChunkDb                      Cleaning up temp files...
[    1, 9,07:12:22.704,INF] TFChunkDb                      Done cleaning up temp files.
[    1, 9,07:12:22.714,INF] Epochmanager                   No epoch checkpoint. Scanning log backwards for most recent epoch...
[    1, 9,07:12:22.717,INF] Epochmanager                   Done scanning log backwards for most recent epoch.
[    1, 9,07:12:22.722,DBG] KurrentDB                      LastEventNumber statically allotted 100,000 Entries
[    1, 9,07:12:22.723,DBG] KurrentDB                      Metadata statically allotted 100,000 Entries
[    1, 9,07:12:22.730,INF] DynamicCacheManager            Cache cache-StreamInfo-LastEventNumber capacity initialized to 100,000 Entries
[    1, 9,07:12:22.731,INF] DynamicCacheManager            Cache cache-StreamInfo-Metadata capacity initialized to 100,000 Entries
[    1, 9,07:12:22.731,INF] DynamicCacheManager            Cache cache-StreamInfo capacity initialized to 200,000 Entries
[    1, 9,07:12:22.731,INF] DynamicCacheManager            Cache cache capacity initialized to 200,000 Entries
[    1, 8,07:12:22.731,INF] ClusterVNodeController         ========== [0.0.0.0:2113] SYSTEM INIT...
[    1,18,07:12:22.739,INF] IndexCommitter                 TableIndex initialization...
[    1, 8,07:12:22.741,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Service 'StorageReaderService' initialized.
[    1,18,07:12:22.742,INF] IndexCommitter                 ReadIndex building...
[    1,18,07:12:22.744,DBG] IndexCommitter                 ReadIndex rebuilding done: total processed 0 records, time elapsed: 00:00:00.0010614.
[    1,18,07:12:22.744,DBG] IndexCommitter                 Initializing the StreamExistenceFilter. The filter can be disabled by setting the configuration option "StreamExistenceFilterSize" to 0
[    1, 8,07:12:22.744,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Service 'KestrelHttpService' initialized.
[    1, 8,07:12:22.744,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Service 'ReplicationTrackingService' initialized.
[    1, 8,07:12:22.744,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Service 'StorageWriterService' initialized.
[    1,18,07:12:22.745,DBG] StreamExistenceFilter          streamExistenceFilter rebuilding started from checkpoint: -1 (0xFFFFFFFFFFFFFFFF).
[    1,18,07:12:22.749,INF] LogV2StreamExistenceFilterInitializer Initializing from index
[    1,18,07:12:22.753,INF] LogV2StreamExistenceFilterInitializer Found table 82515fc2-bb17-4d27-96be-901b0ab48331. Type: KurrentDB.Core.Index.HashListMemTable. Count: 0. Version: 4
[    1,18,07:12:22.758,INF] LogV2StreamExistenceFilterInitializer Initializing from log starting at 0
[    1,18,07:12:22.758,DBG] StreamExistenceFilter          streamExistenceFilter rebuilding done: total processed 0 records, time elapsed: 00:00:00.0130187.
[    1,18,07:12:22.758,DBG] IndexCommitter                 StreamExistenceFilter initialized. Time elapsed: 00:00:00.0143751.
[    1, 8,07:12:22.753,INF] GossipServiceBase              CLUSTER HAS CHANGED 
Old:
["Priority: 0 VND {e5d74f14-8f6a-4814-a8bf-801a02b65430}  [Unknown, Unspecified/127.0.0.1:1112, n/a, n/a, n/a, Unspecified/127.0.0.1:2113, (ADVERTISED: HTTP::0, TCP::0), Version: 25.1.0.2664] -1/0/0/E-1@-1:{00000000-0000-0000-0000-000000000000} | 2026-08-09 07:12:22.736"]
New:
["Priority: 0 VND {e5d74f14-8f6a-4814-a8bf-801a02b65430}  [Unknown, Unspecified/127.0.0.1:1112, n/a, n/a, n/a, Unspecified/127.0.0.1:2113, (ADVERTISED: HTTP::0, TCP::0), Version: 25.1.0.2664] -1/0/0/E-1@-1:{00000000-0000-0000-0000-000000000000} | 2026-08-09 07:12:22.736"]
[    1,19,07:12:22.779,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Service 'StorageChaser' initialized.
[    1,19,07:12:22.782,INF] ClusterVNodeController         ========== [0.0.0.0:2113] SYSTEM START...
[    1,19,07:12:22.784,INF] ClusterVNodeController         ========== [0.0.0.0:2113] IS UNKNOWN...
[    1,15,07:12:22.832,DBG] PersistentSubscriptionService  Persistent subscriptions received state change to Unknown. Stopping listening
[    1,19,07:12:22.832,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [MainQueue]: SystemStart - 52ms. Q: 0/9. KurrentDB.Core.Messages.SystemMessage+SystemStart.
[    1,15,07:12:22.832,DBG] PersistentSubscriptionService  Persistent Subscriptions have been stopped.
[    1,19,07:12:22.832,INF] ClusterVNode                   Current version of KurrentDB is : 25.1.0.2664 
[    1, 9,07:12:22.845,WRN] Kestrel                        Overriding address(es) 'http://*:8080'. Binding to endpoints defined via IConfiguration and/or UseKestrel() instead.
[    1,19,07:12:22.862,INF] ElectionsService               ELECTIONS: STARTING ELECTIONS.
[    1,19,07:12:22.862,INF] ElectionsService               ELECTIONS: (V=0) SHIFT TO LEADER ELECTION.
[    1,19,07:12:22.863,INF] ElectionsService               ELECTIONS: (V=0) VIEWCHANGE FROM [Unspecified/127.0.0.1:2113, {e5d74f14-8f6a-4814-a8bf-801a02b65430}].
[    1,19,07:12:22.863,INF] ElectionsService               ELECTIONS: (V=0) MAJORITY OF VIEWCHANGE.
[    1,19,07:12:22.863,INF] ElectionsService               ELECTIONS: (V=0) SHIFT TO PREPARE PHASE.
[    1,19,07:12:22.864,INF] ElectionsService               ELECTIONS: (V=0) PREPARE_OK FROM [Unspecified/127.0.0.1:2113,{e5d74f14-8f6a-4814-a8bf-801a02b65430}](L=-1,W=0,C=0,E-1@-1:{00000000-0000-0000-0000-000000000000} (L={00000000-0000-0000-0000-000000000000}),Priority=0).
[    1,19,07:12:22.864,INF] ElectionsService               ELECTIONS: (V=0) SHIFT TO REG_LEADER.
[    1,19,07:12:22.866,DBG] ElectionsService               ELECTIONS: (V=0) Proposing node: {e5d74f14-8f6a-4814-a8bf-801a02b65430} as best leader candidate
[    1,19,07:12:22.868,INF] ElectionsService               ELECTIONS: (V=0) SENDING PROPOSAL CANDIDATE: [Unspecified/127.0.0.1:2113,{e5d74f14-8f6a-4814-a8bf-801a02b65430}](L=-1,W=0,C=0,E-1@-1:{00000000-0000-0000-0000-000000000000} (L={00000000-0000-0000-0000-000000000000}),Priority=0), PROPOSAL NUMBER 0, ME: [Unspecified/127.0.0.1:2113,{e5d74f14-8f6a-4814-a8bf-801a02b65430}](L=-1,W=0,C=0,E-1@-1:{00000000-0000-0000-0000-000000000000} (L={00000000-0000-0000-0000-000000000000}),Priority=0).
[    1,19,07:12:22.869,INF] ElectionsService               ELECTIONS: (V=0) ACCEPT FROM [Unspecified/127.0.0.1:2113,{e5d74f14-8f6a-4814-a8bf-801a02b65430}] M=[Unspecified/127.0.0.1:2113,{e5d74f14-8f6a-4814-a8bf-801a02b65430}]).
[    1,19,07:12:22.869,INF] ElectionsService               ELECTIONS: (V=0) DONE. ELECTED LEADER = [Unspecified/127.0.0.1:2113,{e5d74f14-8f6a-4814-a8bf-801a02b65430}](L=-1,W=0,C=0,E-1@-1:{00000000-0000-0000-0000-000000000000} (L={00000000-0000-0000-0000-000000000000}),Priority=0). ME=[Unspecified/127.0.0.1:2113,{e5d74f14-8f6a-4814-a8bf-801a02b65430}](L=-1,W=0,C=0,E-1@-1:{00000000-0000-0000-0000-000000000000} (L={00000000-0000-0000-0000-000000000000}),Priority=0).
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'OtlpExporter' initialized.
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'ConnectedSubsystems' initialized.
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'UserCertificates' initialized.
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'LogsEndpoint' initialized.
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'EncryptionAtRest' initialized.
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'AutoScavenge' initialized.
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'TcpApi' initialized.
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'Connectors' initialized.
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'SchemaRegistry' initialized.
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'APIV2' initialized.
[    1,19,07:12:22.869,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'SecondaryIndexes' initialized.
[    1,19,07:12:22.871,INF] InaugurationManager            InaugurationManager in state (Unknown, Idle): RECEIVED ElectionsDone. currentEpochNumber 0.
[    1,19,07:12:22.871,INF] GossipServiceBase              CLUSTER HAS CHANGED Elections Done
Old:
["Priority: 0 VND {e5d74f14-8f6a-4814-a8bf-801a02b65430}  [Unknown, Unspecified/127.0.0.1:1112, n/a, n/a, n/a, Unspecified/127.0.0.1:2113, (ADVERTISED: HTTP::0, TCP::0), Version: 25.1.0.2664] -1/0/0/E-1@-1:{00000000-0000-0000-0000-000000000000} | 2026-08-09 07:12:22.831"]
New:
["Priority: 0 VND {e5d74f14-8f6a-4814-a8bf-801a02b65430}  [Leader, Unspecified/127.0.0.1:1112, n/a, n/a, n/a, Unspecified/127.0.0.1:2113, (ADVERTISED: HTTP::0, TCP::0), Version: 25.1.0.2664] -1/0/0/E-1@-1:{00000000-0000-0000-0000-000000000000} | 2026-08-09 07:12:22.871"]
[    1,19,07:12:22.872,INF] ClusterVNodeController         ========== [0.0.0.0:2113] PRE-LEADER STATE, WAITING FOR CHASER TO CATCH UP...
[    1,19,07:12:22.872,INF] InaugurationManager            InaugurationManager in state (Unknown, Idle): RECEIVED BecomePreLeader. Starting inauguration process with correlation id 6372213e-7163-44f6-8e53-fa5543365527. Waiting for chaser.
[    1,16,07:12:22.873,DBG] PersistentSubscriptionService  Persistent subscriptions received state change to PreLeader. Stopping listening
[    1,16,07:12:22.873,DBG] PersistentSubscriptionService  Persistent Subscriptions have been stopped.
[    1,25,07:12:22.873,DBG] ProjectionsSubsystem           PROJECTIONS SUBSYSTEM: Not stopping because subsystem is not in a started state. Current Subsystem state: Ready
[    1,25,07:12:22.877,INF] InaugurationManager            InaugurationManager in state (PreLeader, WaitingForChaser): RECEIVED ChaserCaughtUp. RESPONDING with WriteEpoch. currentEpochNumber 0.
[    1,16,07:12:22.931,DBG] TFChunk                        CACHED TFChunk #0-0 (chunk-000000.000000) in 00:00:00.0252372.
[    1,16,07:12:22.949,DBG] Epochmanager                   === Writing E0@0:{de0cacef-d2e2-45b4-9e2a-b8af37eb8dc5} (previous epoch at -1). L={e5d74f14-8f6a-4814-a8bf-801a02b65430}.
[    1,16,07:12:22.971,INF] InaugurationManager            InaugurationManager in state (PreLeader, WritingEpoch): RECEIVED EpochWritten. RESPONDING with EnablePreLeaderReplication.
[    1,16,07:12:22.971,INF] InaugurationManager            InaugurationManager in state (PreLeader, WaitingForConditions): Replication DONE: 548/1. Remaining: 0.
[    1,16,07:12:22.971,INF] InaugurationManager            InaugurationManager in state (PreLeader, WaitingForConditions): Index IN PROGRESS: -1/1. Remaining: 2.
[    1, 9,07:12:22.979,DBG] Epochmanager                   === Cached new Last Epoch E0@0:{de0cacef-d2e2-45b4-9e2a-b8af37eb8dc5} (previous epoch at -1) L={e5d74f14-8f6a-4814-a8bf-801a02b65430}.
[    1,16,07:12:23.088,INF] InaugurationManager            InaugurationManager in state (PreLeader, WaitingForConditions): Replication DONE: 548/1. Remaining: 0.
[    1,16,07:12:23.088,INF] InaugurationManager            InaugurationManager in state (PreLeader, WaitingForConditions): Index DONE: 231/1. Remaining: 0.
[    1,16,07:12:23.088,INF] InaugurationManager            InaugurationManager in state (PreLeader, WaitingForConditions): RECEIVED IndexedTo. RESPONDING with BecomeLeader. Correlation id 6372213e-7163-44f6-8e53-fa5543365527
[    1,16,07:12:23.089,INF] ClusterVNodeController         ========== [0.0.0.0:2113] IS LEADER... SPARTA!
[    1, 9,07:12:23.090,DBG] PersistentSubscriptionService  Persistent subscriptions Became Leader so now handling subscriptions
[    1,16,07:12:23.093,DBG] StorageScavenger               Searching for incomplete scavenges on node 127.0.0.1:2113.
[    1, 9,07:12:23.094,INF] ProjectionsSubsystem           PROJECTIONS SUBSYSTEM: Starting components for Instance: 0431ab2d-ed8f-43bf-8524-84fcbc213a6b
[    1, 9,07:12:23.095,DBG] ProjectionCoreCoordinator      PROJECTIONS: Projection Core Coordinator component starting. Correlation: 0431ab2d-ed8f-43bf-8524-84fcbc213a6b
[    1, 9,07:12:23.096,DBG] ProjectionCoreCoordinator      PROJECTIONS: Starting Projections Core Coordinator
[    1, 9,07:12:23.097,DBG] ProjectionManager              PROJECTIONS: Starting Projections Manager. Correlation: 0431ab2d-ed8f-43bf-8524-84fcbc213a6b
[    1, 9,07:12:23.098,DBG] ProjectionManager              PROJECTIONS: Reading Existing Projections from $projections-$all
[    1, 9,07:12:23.104,DBG] ProjectionsSubsystem           PROJECTIONS SUBSYSTEM: Component 'ProjectionManager' started for Instance: 0431ab2d-ed8f-43bf-8524-84fcbc213a6b
[    1,16,07:12:23.104,DBG] NodeLifetimeService            ConnectorsController node state changed from Unknown to Leader
[    1,16,07:12:23.105,DBG] NodeLifetimeService            ConnectorsStateProjection node state changed from Unknown to Leader
[    1,25,07:12:23.107,DBG] LeaderNodeBackgroundService    ConnectorsController node leadership assigned, running...
[    1, 9,07:12:23.107,DBG] LeaderNodeBackgroundService    ConnectorsStateProjection node leadership assigned, running...
[    1, 8,07:12:23.125,DBG] ProjectionCoreCoordinator      PROJECTIONS: SubComponent Started: EventReaderCoreService
[    1, 8,07:12:23.125,DBG] ProjectionCoreCoordinator      PROJECTIONS: SubComponent Started: EventReaderCoreService
[    1, 8,07:12:23.125,DBG] ProjectionCoreCoordinator      PROJECTIONS: SubComponent Started: ProjectionCoreService
[    1, 8,07:12:23.125,DBG] ProjectionCoreCoordinator      PROJECTIONS: SubComponent Started: EventReaderCoreService
[    1, 8,07:12:23.125,DBG] ProjectionCoreCoordinator      PROJECTIONS: SubComponent Started: ProjectionCoreService
[    1, 8,07:12:23.125,DBG] ProjectionCoreCoordinator      PROJECTIONS: SubComponent Started: ProjectionCoreService
[    1, 8,07:12:23.125,DBG] ProjectionsSubsystem           PROJECTIONS SUBSYSTEM: Component 'ProjectionCoreCoordinator' started for Instance: 0431ab2d-ed8f-43bf-8524-84fcbc213a6b
[    1, 8,07:12:23.125,INF] ProjectionsSubsystem           PROJECTIONS SUBSYSTEM: All components started for Instance: 0431ab2d-ed8f-43bf-8524-84fcbc213a6b
[    1, 8,07:12:23.125,INF] ClusterVNodeController         ========== [0.0.0.0:2113] Sub System 'Projections' initialized.
[    1, 8,07:12:23.126,DBG] DuckDBProjectorBackgroundService DuckDBProjector system is ready
[    1,15,07:12:23.140,DBG] PersistentSubscriptionService  Persistent Subscriptions have been started.
[    1,15,07:12:23.149,DBG] ProjectionManager              PROJECTIONS: No projections were found in $projections-$all, starting from empty stream
[    1,25,07:12:23.209,DBG] SystemProcessor                Processor ConnectorsStateProjection Activating { Filter = { Scope = Stream, Type = Regex, Expression = ^\$connectors\/([^\/]+)$ } }
[    1, 8,07:12:23.212,DBG] DuckDBProjector                [SchemaProjection] Loaded checkpoint null (Id: 1)
[    1,18,07:12:23.218,INF] SystemProcessor                Processor ConnectorsStateProjection Running
[    1, 8,07:12:23.221,INF] SecondaryIndexBuilder          Starting indexing subscription from C:0/P:0
[    1, 8,07:12:23.222,DBG] InMemoryBus                    SLOW BUS MSG [MainBus]: SystemReady - 97ms.
[    1, 8,07:12:23.222,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [MainQueue]: SubSystemInitialized - 97ms. Q: 0/17. KurrentDB.Core.Messages.SystemMessage+SubSystemInitialized.
[    1, 8,07:12:23.223,DBG] StorageScavenger               Setting max age for the $scavenges stream to 30.00:00:00.
[    1, 8,07:12:23.223,DBG] StorageScavenger               Setting $ops read permission for the $scavenges stream
[    1, 8,07:12:23.225,DBG] StorageScavenger               No incomplete scavenges found on node 127.0.0.1:2113.
[    1,15,07:12:23.243,DBG] SystemStartupTaskService       FixConnectorsControlRegistryStreamName completed
[    1, 8,07:12:23.259,DBG] SystemStartupTaskService       ConfigureConnectorsManagementStreams Stream $connectors-mngt/state-projection configured
[    1,18,07:12:23.269,DBG] ProjectionManager              PROJECTIONS: Found the following new projections in $projections-$all: ["$streams","$stream_by_category","$by_category","$by_event_type","$by_correlation_id"]
[    1,16,07:12:23.270,DBG] SystemStartupTaskService       ConfigureConnectorsManagementStreams Stream $connectors-mngt/state-projection/checkpoints configured
[    1,16,07:12:23.270,DBG] SystemStartupTaskService       ConfigureConnectorsManagementStreams completed
[    1,16,07:12:23.272,INF] SystemStartupManager           System startup tasks completed
[    1,18,07:12:23.273,DBG] ProjectionManager              PROJECTIONS: Caught up with projections registration. Next expected version: 5
[    1,19,07:12:23.294,DBG] ProjectionManager              Adding projection 580efa2f-af34-41e8-9366-a81d1ba1c23b@$streams to list
[    1,19,07:12:23.295,DBG] ProjectionManager              Adding projection bcba986a-5d8e-488e-96d4-5fee087a55e9@$by_category to list
[    1,19,07:12:23.296,DBG] ProjectionManager              Adding projection 4de39aa4-03d1-488c-b266-41b40c0e36b7@$by_correlation_id to list
[    1,19,07:12:23.296,DBG] ProjectionManager              Adding projection a44509a0-c6ec-400f-b3ed-444c1c22c8bd@$by_event_type to list
[    1,19,07:12:23.296,DBG] ProjectionManager              Adding projection eb51e6e6-a786-4a62-bfc5-9e0866f1dac5@$stream_by_category to list
[    1,16,07:12:23.434,INF] ProjectionManager              '$by_correlation_id' projection source has been written
[    1,16,07:12:23.435,INF] ProjectionManager              '$by_category' projection source has been written
[    1,16,07:12:23.435,INF] ProjectionManager              '$streams' projection source has been written
[    1,16,07:12:23.435,INF] ProjectionManager              '$stream_by_category' projection source has been written
[    1,16,07:12:23.435,INF] ProjectionManager              '$by_event_type' projection source has been written
[    1, 9,07:12:23.446,INF] ProjectionManager              Enabling '$by_correlation_id' projection
[    1, 9,07:12:23.447,INF] ProjectionManager              Enabling '$by_category' projection
[    1, 9,07:12:23.447,INF] ProjectionManager              Enabling '$streams' projection
[    1, 9,07:12:23.447,INF] ProjectionManager              Enabling '$by_event_type' projection
[    1, 9,07:12:23.447,INF] ProjectionManager              Enabling '$stream_by_category' projection
[    1,18,07:12:23.562,INF] ProjectionManager              '$by_category' projection source has been written
[    1,18,07:12:23.562,INF] ProjectionManager              '$by_correlation_id' projection source has been written
[    1,18,07:12:23.562,INF] ProjectionManager              '$streams' projection source has been written
[    1,18,07:12:23.562,INF] ProjectionManager              '$by_event_type' projection source has been written
[    1,18,07:12:23.562,INF] ProjectionManager              '$stream_by_category' projection source has been written
[    1,43,07:12:23.973,INF] InaugurationManager            InaugurationManager in state (Leader, Idle): IGNORING CheckInaugurationConditions. Not waiting for conditions.
[    1,44,07:12:26.592,DBG] InMemoryBus                    SLOW BUS MSG [SubscriptionsBus]: EventCommitted - 130ms.
[    1,11,07:12:26.593,INF] GcSuspensionMetric             Garbage collection: Long Execution Engine Suspension. Reason: GarbageCollection. Took: 130ms
[    1,45,07:12:28.707,DBG] DefaultIndexProcessor          Secondary index default records committed in 26 ms
[    1,18,07:12:29.908,DBG] InMemoryBus                    SLOW BUS MSG [bus]: UnwrapEnvelopeMessage - 70ms.
[    1,18,07:12:29.908,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #2]: UnwrapEnvelopeMessage - 70ms. Q: 0/0. KurrentDB.Projections.Core.Messaging.UnwrapEnvelopeMessage, Extra Information: ReadAllEventsForward.
[    1,52,07:12:31.729,DBG] DefaultIndexProcessor          Secondary index default records committed in 50 ms
[    1,37,07:12:31.890,DBG] InMemoryBus                    SLOW BUS MSG [bus]: CoreTick - 80ms.
[    1,37,07:12:31.890,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #2]: CoreTick - 81ms. Q: 0/0. KurrentDB.Projections.Core.Messages.ProjectionCoreServiceMessage+CoreTick.
[    1,35,07:12:32.522,DBG] TFChunk                        CACHED TFChunk #1-1 (chunk-000001.000000) in 00:00:00.0000962.
[    1,52,07:12:32.669,DBG] InMemoryBus                    SLOW BUS MSG [bus]: UnwrapEnvelopeMessage - 75ms.
[    1,52,07:12:32.669,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #1]: UnwrapEnvelopeMessage - 75ms. Q: 0/253. KurrentDB.Projections.Core.Messaging.UnwrapEnvelopeMessage, Extra Information: ReadAllEventsForward.
[    1,35,07:12:33.457,DBG] InMemoryBus                    SLOW BUS MSG [bus]: CoreTick - 80ms.
[    1,35,07:12:33.457,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #0]: CoreTick - 80ms. Q: 0/4. KurrentDB.Projections.Core.Messages.ProjectionCoreServiceMessage+CoreTick.
[    1,36,07:12:33.457,DBG] InMemoryBus                    SLOW BUS MSG [bus]: CommittedEventReceived - 80ms.
[    1,36,07:12:33.457,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #1]: CommittedEventReceived - 80ms. Q: 59/65. C:18057101/P:18057101.
[    1,39,07:12:33.641,DBG] DefaultIndexProcessor          Secondary index default records committed in 111 ms
[    1,19,07:12:34.263,DBG] InMemoryBus                    SLOW BUS MSG [bus]: UnwrapEnvelopeMessage - 69ms.
[    1,45,07:12:34.263,DBG] InMemoryBus                    SLOW BUS MSG [bus]: CommittedEventDistributed - 69ms.
[    1,19,07:12:34.263,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #0]: UnwrapEnvelopeMessage - 69ms. Q: 1/35. KurrentDB.Projections.Core.Messaging.UnwrapEnvelopeMessage, Extra Information: ReadAllEventsForward.
[    1,45,07:12:34.263,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #2]: CommittedEventDistributed - 69ms. Q: 212/212. KurrentDB.Projections.Core.Messages.ReaderSubscriptionMessage+CommittedEventDistributed.
[    1,40,07:12:34.263,DBG] InMemoryBus                    SLOW BUS MSG [bus]: UnwrapEnvelopeMessage - 69ms.
[    1,51,07:12:34.263,DBG] InMemoryBus                    SLOW BUS MSG [output bus]: CommittedEventDistributed - 69ms.
[    1,40,07:12:34.263,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #1]: UnwrapEnvelopeMessage - 69ms. Q: 1/452. KurrentDB.Projections.Core.Messaging.UnwrapEnvelopeMessage, Extra Information: ReadAllEventsForward.
[    1,51,07:12:34.263,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #0 output]: CommittedEventDistributed - 69ms. Q: 182/219. KurrentDB.Projections.Core.Messages.ReaderSubscriptionMessage+CommittedEventDistributed.
[    1,41,07:12:34.263,DBG] InMemoryBus                    SLOW BUS MSG [SubscriptionsBus]: EventCommitted - 69ms.
[    1,38,07:12:34.414,DBG] TableIndex                     Switching MemTable, currently: 2 awaiting tables.
[    1,49,07:12:34.416,DBG] TableIndex                     Awaiting tables queue size is: 2.
[    1,49,07:12:34.419,DBG] FileStreamPersistence          Zeroing bloom filter...
[    1,49,07:12:34.419,DBG] FileStreamPersistence          Done zeroing bloom filter
[    1,49,07:12:34.419,INF] FileStreamPersistence          Writing header and expanding file...
[    1,49,07:12:34.421,INF] FileStreamPersistence          Wrote header and expanded file
[    1,49,07:12:34.570,DBG] PTable                         Cached 65536 index midpoints to PTable
[    1,49,07:12:34.592,DBG] PTable                         Dumped MemTable [82515fc2-bb17-4d27-96be-901b0ab48331, 1000236 entries] in 00:00:00.1740923.
[    1,49,07:12:34.594,DBG] PTable                         Loading and Verification of PTable 'aeb21038-b551-41ba-9f43-d815547468b7' started...
[    1,49,07:12:34.692,INF] FileStreamPersistence          Reading persisted bloom filter /var/lib/kurrentdb/index/aeb21038-b551-41ba-9f43-d815547468b7.bloomfilter of size 266,816 bytes into memory...
[    1,49,07:12:34.692,INF] FileStreamPersistence          Read persisted bloom filter /var/lib/kurrentdb/index/aeb21038-b551-41ba-9f43-d815547468b7.bloomfilter into memory. Took 00:00:00.0000587. 0.00 MB/s
[    1,49,07:12:34.693,DBG] FileStreamPersistence          Verifying bloom filter...
[    1,49,07:12:34.694,DBG] FileStreamPersistence          Done verifying bloom filter
[    1,49,07:12:34.695,DBG] PTable                         Loading PTable (Version: 4) 'aeb21038-b551-41ba-9f43-d815547468b7' (1000236 entries, cache depth 16) done in 00:00:00.1003865.
[    1,49,07:12:34.702,DBG] TableIndex                     There are now 1 awaiting tables.
[    1,49,07:12:34.702,DBG] TableIndex                     Awaiting tables queue size is: 1.
[    1,16,07:12:35.232,DBG] InMemoryBus                    SLOW BUS MSG [bus]: CommittedEventDistributed - 216ms.
[    1,16,07:12:35.232,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #0]: CommittedEventDistributed - 216ms. Q: 199/452. KurrentDB.Projections.Core.Messages.ReaderSubscriptionMessage+CommittedEventDistributed.
[    1,46,07:12:35.232,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [StorageReaderQueue #4]: ReadAllEventsForward - 217ms. Q: 0/1. ReadAllEventsForward InternalCorrId: 5b0aaad3-b6ea-4b4d-80bd-b6d929b16124, CorrelationId: 5b0aaad3-b6ea-4b4d-80bd-b6d929b16124, User: system, Envelope: { KurrentDB.Projections.Core.Messaging.PublishToWrapEnvelop }, Expires: 08/09/2026 07:12:45, PreparePosition: 0, CommitPosition: 281460857, MaxCount: 250, ResolveLinkTos: True, RequireLeader: False, ValidationTfLastCommitPosition: , LongPollTimeout: , ReplyOnExpired: False.
[    1,46,07:12:35.486,DBG] DefaultIndexProcessor          Secondary index default records committed in 40 ms
[    1, 9,07:12:36.045,DBG] InMemoryBus                    SLOW BUS MSG [bus]: CoreTick - 70ms.
[    1, 9,07:12:36.046,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #1]: CoreTick - 70ms. Q: 1/2. KurrentDB.Projections.Core.Messages.ProjectionCoreServiceMessage+CoreTick.
[    1,18,07:12:36.624,DBG] TFChunk                        CACHED TFChunk #2-2 (chunk-000002.000000) in 00:00:00.0000565.
[    1,25,07:12:36.625,DBG] TFChunk                        UNCACHING TFChunk #0-0 (chunk-000000.000000).
[    1,25,07:12:36.650,DBG] TFChunk                        UNCACHED TFChunk #0-0 (chunk-000000.000000).
[    1,51,07:12:37.024,DBG] InMemoryBus                    SLOW BUS MSG [bus]: CoreTick - 234ms.
[    1,51,07:12:37.024,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #1]: CoreTick - 234ms. Q: 0/2. KurrentDB.Projections.Core.Messages.ProjectionCoreServiceMessage+CoreTick.
[    1,34,07:12:37.024,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [StorageReaderQueue #2]: ReadAllEventsForward - 237ms. Q: 0/3. ReadAllEventsForward InternalCorrId: 336cc834-0f98-462e-bea9-e9179ceb28e1, CorrelationId: 336cc834-0f98-462e-bea9-e9179ceb28e1, User: system, Envelope: { KurrentDB.Projections.Core.Messaging.PublishToWrapEnvelop }, Expires: 08/09/2026 07:12:46, PreparePosition: 0, CommitPosition: 339659244, MaxCount: 250, ResolveLinkTos: True, RequireLeader: False, ValidationTfLastCommitPosition: , LongPollTimeout: , ReplyOnExpired: False.
[    1, 9,07:12:37.025,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [StorageReaderQueue #3]: ReadAllEventsForward - 236ms. Q: 0/1. ReadAllEventsForward InternalCorrId: f1eb2042-fdb9-4346-a85f-f0ea08ba5ccf, CorrelationId: f1eb2042-fdb9-4346-a85f-f0ea08ba5ccf, User: system, Envelope: { KurrentDB.Projections.Core.Messaging.PublishToWrapEnvelop }, Expires: 08/09/2026 07:12:46, PreparePosition: 0, CommitPosition: 340574355, MaxCount: 250, ResolveLinkTos: True, RequireLeader: False, ValidationTfLastCommitPosition: , LongPollTimeout: , ReplyOnExpired: False.
[    1,11,07:12:37.025,INF] GcSuspensionMetric             Garbage collection: Long Execution Engine Suspensions omitted. 7 suspensions took 94ms each on average
[    1,11,07:12:37.025,INF] GcSuspensionMetric             Garbage collection: Long Execution Engine Suspension. Reason: GarbageCollection. Took: 234ms
[    1,51,07:12:37.938,DBG] DefaultIndexProcessor          Secondary index default records committed in 47 ms
[    1,49,07:12:39.902,DBG] DefaultIndexProcessor          Secondary index default records committed in 143 ms
[    1,34,07:12:40.010,DBG] TableIndex                     Switching MemTable, currently: 2 awaiting tables.
[    1,34,07:12:40.010,DBG] TableIndex                     Awaiting tables queue size is: 2.
[    1,34,07:12:40.010,DBG] FileStreamPersistence          Zeroing bloom filter...
[    1,34,07:12:40.010,DBG] FileStreamPersistence          Done zeroing bloom filter
[    1,34,07:12:40.010,INF] FileStreamPersistence          Writing header and expanding file...
[    1,34,07:12:40.012,INF] FileStreamPersistence          Wrote header and expanded file
[    1,34,07:12:40.098,DBG] PTable                         Cached 65536 index midpoints to PTable
[    1,34,07:12:40.116,DBG] PTable                         Dumped MemTable [af53ebe4-881a-4d07-b5cf-7dcaaa378eff, 1000027 entries] in 00:00:00.1060748.
[    1,34,07:12:40.116,DBG] PTable                         Loading and Verification of PTable '105f3cca-b9c9-4d8f-8261-32574200dddb' started...
[    1,34,07:12:40.177,INF] FileStreamPersistence          Reading persisted bloom filter /var/lib/kurrentdb/index/105f3cca-b9c9-4d8f-8261-32574200dddb.bloomfilter of size 266,752 bytes into memory...
[    1,34,07:12:40.177,INF] FileStreamPersistence          Read persisted bloom filter /var/lib/kurrentdb/index/105f3cca-b9c9-4d8f-8261-32574200dddb.bloomfilter into memory. Took 00:00:00.0000458. 0.00 MB/s
[    1,34,07:12:40.177,DBG] FileStreamPersistence          Verifying bloom filter...
[    1,34,07:12:40.178,DBG] FileStreamPersistence          Done verifying bloom filter
[    1,34,07:12:40.178,DBG] PTable                         Loading PTable (Version: 4) '105f3cca-b9c9-4d8f-8261-32574200dddb' (1000027 entries, cache depth 16) done in 00:00:00.0617698.
[    1,34,07:12:40.211,DBG] PTable                         PTables merge started (specialized for <= 2 tables).
[    1,34,07:12:40.212,DBG] FileStreamPersistence          Zeroing bloom filter...
[    1,34,07:12:40.213,DBG] FileStreamPersistence          Done zeroing bloom filter
[    1,34,07:12:40.213,INF] FileStreamPersistence          Writing header and expanding file...
[    1,34,07:12:40.217,INF] FileStreamPersistence          Wrote header and expanded file
[    1,37,07:12:40.518,DBG] TFChunk                        CACHED TFChunk #3-3 (chunk-000003.000000) in 00:00:00.0000757.
[    1,52,07:12:40.518,DBG] TFChunk                        UNCACHING TFChunk #1-1 (chunk-000001.000000).
[    1,52,07:12:40.538,DBG] TFChunk                        UNCACHED TFChunk #1-1 (chunk-000001.000000).
[    1,34,07:12:40.620,DBG] PTable                         Cached 65536 index midpoints to PTable
[    1,34,07:12:40.648,DBG] PTable                         PTables merge finished in 00:00:00.4349009 ([1000236, 1000027] entries merged into 2000263).
[    1,34,07:12:40.648,DBG] PTable                         Loading and Verification of PTable 'db8be27f-7ab3-4472-947d-99d072c16e17' started...
[    1,36,07:12:40.813,DBG] InMemoryBus                    SLOW BUS MSG [bus]: CoreTick - 81ms.
[    1, 9,07:12:40.813,DBG] InMemoryBus                    SLOW BUS MSG [bus]: CoreTick - 81ms.
[    1, 9,07:12:40.813,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #1]: CoreTick - 81ms. Q: 0/2. KurrentDB.Projections.Core.Messages.ProjectionCoreServiceMessage+CoreTick.
[    1,36,07:12:40.813,DBG] QueuedHandlerThreadPool        SLOW QUEUE MSG [Projection Core #0]: CoreTick - 81ms. Q: 0/1. KurrentDB.Projections.Core.Messages.ProjectionCoreServiceMessage+CoreTick.
[    1,11,07:12:40.875,INF] GcSuspensionMetric             Start of full blocking garbage collection at 08/09/2026 07:12:40. GC: #27. Generation: 2. Reason: InducedButNotForcedAsBlocking. Type: BlockingOutsideBackgroundGC.
[    1,34,07:12:40.875,INF] FileStreamPersistence          Reading persisted bloom filter /var/lib/kurrentdb/index/db8be27f-7ab3-4472-947d-99d072c16e17.bloomfilter of size 533,504 bytes into memory...
[    1,11,07:12:40.876,INF] GcSuspensionMetric             End of full blocking garbage collection at 08/09/2026 07:12:40. GC: #27. Took: 27ms
[    1,34,07:12:40.876,INF] FileStreamPersistence          Read persisted bloom filter /var/lib/kurrentdb/index/db8be27f-7ab3-4472-947d-99d072c16e17.bloomfilter into memory. Took 00:00:00.0001371. 0.00 MB/s
[    1,34,07:12:40.876,DBG] FileStreamPersistence          Verifying bloom filter...
[    1,34,07:12:40.878,DBG] FileStreamPersistence          Done verifying bloom filter
[    1,34,07:12:40.878,DBG] PTable                         Loading PTable (Version: 4) 'db8be27f-7ab3-4472-947d-99d072c16e17' (2000263 entries, cache depth 16) done in 00:00:00.2300006.
[    1,34,07:12:40.887,DBG] TableIndex                     There are now 1 awaiting tables.
[    1,34,07:12:40.887,DBG] TableIndex                     Awaiting tables queue size is: 1.