Earlier last week I was working with Percona Binlog Server, Percona’s tool for archiving the MySQL binary log (binlog) outside of a traditional replica and exploring what it can do. In this post I’ll share my hands-on experience setting it up with Docker against a MySQL 8.4 source: the config, the two replication modes it supports (position and GTID), and the commands that make it useful day to day.
Percona Binlog Server Lab setup:
- A MySQL 8.4 source
- Docker on my local machine (a Mac)
What Is Percona Binlog Server?
Percona Binary Log Server (binlog_server) is a command-line utility that connects to a remote MySQL or Percona server as a replication client to stream and archive binary log events. It can stream the binary log off to durable storage of your choice. Point it at a file backend, an S3 bucket, whatever you need, and it’ll keep pulling continuously, surviving reconnects, and letting you search what it’s collected later by timestamp or by GTID.
Trying out PBS
I am using the details available on Percona’s PBS github repo: https://github.com/Percona-Lab/percona-binlog-server. Following are my steps to setup and use PBS.
I have docker configured on my mac and I could start setting-up the percona-binlog-server.
docker pull perconalab/percona-binlog-server:latest
docker run --rm perconalab/percona-binlog-server:latest binlog_server versionVersion resolved to 0.4.1 at the time of writing, latest is a moving target, so worth checking each time.
Preparing the MySQL Source Server
I have a MySQL 8.4 instance, with binlog already on. We also have gtid_mode and enforice_gtid_consistency configured ON. Before pointing Percona Binlog Server at it, I confirmed the basics:
SHOW VARIABLES LIKE 'log_bin';
SHOW VARIABLES LIKE 'gtid_mode';
SHOW VARIABLES LIKE 'enforce_gtid_consistency';
SHOW VARIABLES LIKE 'server_id';Created a dedicated replication account, since binlog_server talks to the source exactly like a replica would:
CREATE USER 'rpl_user'@'%' IDENTIFIED BY '';
GRANT REPLICATION SLAVE ON *.* TO 'rpl_user'@'%';
FLUSH PRIVILEGES; Setting Up Percona Binlog Server with Docker on macOS
Prepare folders to hold data and config for PBS
mkdir -p ~/binlog-server/data
mkdir -p ~/binlog-server/logsThe config.json file
{
"logger": { "level": "info", "file": "" },
"connection": {
"host": "10.30.50.220",
"port": 3306,
"user": "rpl_user",
"password": "",
"connect_timeout": 20,
"read_timeout": 60,
"write_timeout": 60
},
"replication": {
"server_id": 9001,
"idle_time": 30,
"verify_checksum": true,
"mode": "position"
},
"storage": {
"backend": "file",
"uri": "file:///var/lib/binlog-server/data"
}
} That’s it, once the directories and config is ready we can do a test run for PBS to collect binary logs onetime:
docker run --rm \
-v ~/binlog-server/config.json:/etc/binlog-server/config.json:ro \
-v ~/binlog-server/data:/var/lib/binlog-server/data \
-v ~/binlog-server/logs:/var/log/binlog-server \
perconalab/percona-binlog-server:latest \
binlog_server fetch /etc/binlog-server/config.json$ docker run --rm \
> -v ~/binlog-server/config.json:/etc/binlog-server/config.json:ro \
> -v ~/binlog-server/data:/var/lib/binlog-server/data \
> -v ~/binlog-server/logs:/var/log/binlog-server \
> perconalab/percona-binlog-server:latest \
> binlog_server fetch /etc/binlog-server/config.json
[ ] "binlog_server" started with the following command line arguments:
[ ] "fetch" "/etc/binlog-server/config.json"
[ ] reading configuration from the JSON file.
[ ] logging level set to "info"
[ ] application version: 0.4.1
[ ] 'fetch' operation mode specified
[ info] set custom handlers for SIGINT and SIGTERM signals
[ info] keyring configuration options are not specified
[ info] binlog storage backend type: file
[ info] binlog storage backend URI (masked): file:///var/lib/binlog-server/data
[ info] mysql connection string: rpl_user@10.30.50.220:3306 (password ***hidden***)
[ info] mysql connect timeout (seconds): 20
[ info] mysql read timeout (seconds): 60
[ info] mysql write timeout (seconds): 60
[ info] mysql replication server id: 9001
[ info] mysql replication idle time (seconds): 30
[ info] mysql replication checksum verification: true
[ info] mysql replication mode: position
[ info] created binlog storage with the following backend: local filesystem - /var/lib/binlog-server/data
[ info] binlog storage initialized in position mode
[ info] binlog storage initialized on an empty directory
[ info] storage keyring status: keyring is not initialized
[ info] storage active KEK: active KEK is not set
[ info] storage encryption format: encryption format is not set
[ info] initialized mysql client library
[ info] mysql client version: 8.4.10-10
[ info] established connection to mysql server
[ info] mysql server version: 8.4.11-11
[ info] mysql protocol version: 10
[ info] mysql server connection info: 10.30.50.220 via TCP/IP
[ info] mysql connection character set: utf8mb4
[ info] switched to replication (checksum enabled, position mode)
[ info] replication info (server id 9001, non-blocking, starting from the very beginning)
[ info] event : rotate (artificial)
[ info] event : [info_only] - will not be written to the binary log file
[ info] storage: created a new binlog file: mysql-bin.000001
[ info] event : format_description
[ info] event : previous_gtids_log (ignorable)
[ info] event : gtid_log
[ info] event : query
[ info] event : [end_of_transaction] 7021a5ff-ac64-11f1-bc37-bc24112b899e:1
[ info] event : gtid_log
[ info] event : query
[ info] event : [end_of_transaction] 7021a5ff-ac64-11f1-bc37-bc24112b899e:2
[ info] event : gtid_log
[ info] event : query
[ info] event : [end_of_transaction] 7021a5ff-ac64-11f1-bc37-bc24112b899e:3
[ info] fetched everything and disconnected
[ info] successfully shut down after finishing the requested operationWe can see that both binlog and related metadata files are present and fetched.
$ ls -lhtr ~/binlog-server/data/
total 32
-rw-r--r-- 1 pbs mysql 31B 9 Sep 22:56 metadata.json
-rw-r--r-- 1 pbs mysql 19B 9 Sep 22:56 binlog.index
-rw-r--r-- 1 pbs mysql 866B 9 Sep 22:56 mysql-bin.000001
-rw-r--r-- 1 pbs mysql 125B 9 Sep 22:56 mysql-bin.000001.jsonNow that we verified everything works, let’s start the Percona Binlog Server in PULL mode to continuously fetch the binlogs.
docker run -d --name binlog-server \
--restart unless-stopped \
--stop-signal SIGTERM \
--stop-timeout 120 \
-v ~/binlog-server/config.json:/etc/binlog-server/config.json:ro \
-v ~/binlog-server/data:/var/lib/binlog-server/data \
-v ~/binlog-server/logs:/var/log/binlog-server \
perconalab/percona-binlog-server:latest \
binlog_server pull /etc/binlog-server/config.jsonA few minutes later, data folder had binlog files sitting in it:
$ ls -lhtr
total 589888
-rw-r--r-- 1 pbs mysql 31B 9 Sep 22:56 metadata.json
-rw-r--r-- 1 pbs mysql 101M 9 Sep 23:02 mysql-bin.000001
-rw-r--r-- 1 pbs mysql 134B 9 Sep 23:02 mysql-bin.000001.json
-rw-r--r-- 1 pbs mysql 161M 9 Sep 23:03 mysql-bin.000002
-rw-r--r-- 1 pbs mysql 131B 9 Sep 23:03 mysql-bin.000002.json
-rw-r--r-- 1 pbs mysql 4B 9 Sep 23:03 mysql-bin.000003
-rw-r--r-- 1 pbs mysql 123B 9 Sep 23:03 mysql-bin.000003.json
-rw-r--r-- 1 pbs mysql 57B 9 Sep 23:03 binlog.indexThe docker status shows that the binlog server is running
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0f73436929f1 perconalab/percona-binlog-server:latest "binlog_server pull …" 14 hours ago Up 13 hours binlog-serverPercona Binlog Server Commands
Beyond fetch (one-shot) and pull (continuous, what I’m running), there’s list, to see everything collected so far, and a couple of search commands that make the whole exercise worthwhile. Finding which binlog file covers a moment or a transaction you care about, without grepping through gigabytes by hand. These functions help during recovery practices.
binlog_server list
docker run --rm \
-v ~/binlog-server/config.json:/etc/binlog-server/config.json:ro \
-v ~/binlog-server/data:/var/lib/binlog-server/data:ro \
perconalab/percona-binlog-server:latest \
binlog_server list /etc/binlog-server/config.json | jq
{
"version": 1,
"status": "success",
"result": [
{
"name": "mysql-bin.000001",
"size": 105825820,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000001",
"min_timestamp": "2026-09-09T15:39:08",
"max_timestamp": "2026-09-09T17:32:14"
},
{
"name": "mysql-bin.000002",
"size": 169304503,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000002",
"min_timestamp": "2026-09-09T17:32:14",
"max_timestamp": "2026-09-09T17:32:51"
},
{
"name": "mysql-bin.000003",
"size": 169304257,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000003",
"min_timestamp": "2026-09-09T17:32:51",
"max_timestamp": "2026-09-09T17:33:02"
},
{
"name": "mysql-bin.000004",
"size": 274547106,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000004",
"min_timestamp": "2026-09-09T17:33:02",
"max_timestamp": "2026-09-09T17:44:35"
},
{
"name": "mysql-bin.000005",
"size": 12557,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000005",
"min_timestamp": "2026-09-09T17:44:35",
"max_timestamp": "2026-09-09T18:27:32"
}
]
}binlog_server search_by_timestamp
This PBS command helps identify binary log information at specified timestamp
docker run --rm \
-v ~/binlog-server/config.json:/etc/binlog-server/config.json:ro \
-v ~/binlog-server/data:/var/lib/binlog-server/data:ro \
perconalab/percona-binlog-server:latest \
binlog_server search_by_timestamp /etc/binlog-server/config.json 2026-09-09T22:56:00binlog_server search_by_gtid_set
This PBS command helps identify binary log providing the searched gtid-set.
docker run --rm \
-v ~/binlog-server/config-gtid.json:/etc/binlog-server/config.json:ro \
-v ~/binlog-server/data-gtid:/var/lib/binlog-server/data:ro \
perconalab/percona-binlog-server:latest \
binlog_server search_by_gtid_set /etc/binlog-server/config.json \
7021a5ff-ac64-11f1-bc37-bc24112b899e:2117 | jq
{"version":1,"status":"error","message":"GTID set search is not supported in storages created in position-based replication mode"}oops! We hit the road block here! As the error log clearly says, we cannot search GTID if the replication mode is “position based”. If you note the configuration file created above we had the setting:
"mode": "position"Starting PBS with GTID
Let’s start PBS with GTID and redo the functions that stopped us earlier 🙂
Setting up PBS config
{
"logger": { "level": "info", "file": "" },
"connection": {
"host": "10.30.50.220",
"port": 3306,
"user": "rpl_user",
"password": "",
"connect_timeout": 20,
"read_timeout": 60,
"write_timeout": 60
},
"replication": {
"server_id": 9002,
"idle_time": 30,
"verify_checksum": true,
"mode": "gtid"
},
"storage": {
"backend": "file",
"uri": "file:///var/lib/binlog-server/data"
}
} Note that in the config file we have wwo changes from the original: "mode": "gtid" instead of "position", and a different server_id (9002); since this is a second replication client connecting to the same source at the same time as the original collector, and each one needs its own unique ID.
Starting PBS GTID pull
docker run -d --name binlog-server-gtid \
--restart unless-stopped \
--stop-signal SIGTERM \
--stop-timeout 120 \
-v ~/binlog-server/config-gtid.json:/etc/binlog-server/config.json:ro \
-v ~/binlog-server/data-gtid:/var/lib/binlog-server/data \
-v ~/binlog-server/logs:/var/log/binlog-server \
perconalab/percona-binlog-server:latest \
binlog_server pull /etc/binlog-server/config.jsonYou can see the status of both the docker containers running percona binlog server.
On Source:
mysql> show processlist;
+-----+-----------------+--------------------+------+------------------+-------+-----------------------------------------------------------------+------------------+----------+-----------+---------------+
| Id | User | Host | db | Command | Time | State | Info | Time_ms | Rows_sent | Rows_examined |
+-----+-----------------+--------------------+------+------------------+-------+-----------------------------------------------------------------+------------------+----------+-----------+---------------+
| 6 | event_scheduler | localhost | NULL | Daemon | 10134 | Waiting on empty queue | NULL | 10133872 | 0 | 0 |
| 25 | root | localhost | test | Query | 0 | init | show processlist | 2 | 0 | 0 |
| 222 | rpl_user | 10.222.20.99:62850 | NULL | Binlog Dump | 73 | Source has sent all binlog to replica; waiting for more updates | NULL | 73678 | 0 | 0 |
| 223 | rpl_user | 10.222.20.99:62856 | NULL | Binlog Dump GTID | 49 | Source has sent all binlog to replica; waiting for more updates | NULL | 49704 | 0 | 0 |
+-----+-----------------+--------------------+------+------------------+-------+-----------------------------------------------------------------+------------------+----------+-----------+---------------+Docker status
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a7cfdbe091bd perconalab/percona-binlog-server:latest "binlog_server pull …" 13 hours ago Up 13 hours binlog-server-gtid
0f73436929f1 perconalab/percona-binlog-server:latest "binlog_server pull …" 14 hours ago Up 13 hours binlog-serverbinlog_server list
Now, let’s redo the gtid search but before that let me first show you the difference of “list”… See that the metadata has more information regarding GTID:
docker run --rm \
-v ~/binlog-server/config-gtid.json:/etc/binlog-server/config.json:ro \
-v ~/binlog-server/data-gtid:/var/lib/binlog-server/data:ro \
perconalab/percona-binlog-server:latest \
binlog_server list /etc/binlog-server/config.json | jq
{
"version": 1,
"status": "success",
"result": [
{
"name": "mysql-bin.000001",
"size": 105825820,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000001",
"previous_gtids": "",
"added_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:1-2114",
"min_timestamp": "2026-09-09T15:39:08",
"max_timestamp": "2026-09-09T17:32:14"
},
{
"name": "mysql-bin.000002",
"size": 169304503,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000002",
"previous_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:1-2114",
"added_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:2115-2116",
"min_timestamp": "2026-09-09T17:32:14",
"max_timestamp": "2026-09-09T17:32:51"
},
{
"name": "mysql-bin.000003",
"size": 169304257,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000003",
"previous_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:1-2116",
"added_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:2117",
"min_timestamp": "2026-09-09T17:32:51",
"max_timestamp": "2026-09-09T17:33:02"
},
{
"name": "mysql-bin.000004",
"size": 274547106,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000004",
"previous_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:1-2117",
"added_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:2118",
"min_timestamp": "2026-09-09T17:33:02",
"max_timestamp": "2026-09-09T17:44:35"
},
{
"name": "mysql-bin.000005",
"size": 12557,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000005",
"previous_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:1-2118",
"added_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:2119-2120",
"min_timestamp": "2026-09-09T17:44:35",
"max_timestamp": "2026-09-09T18:27:32"
}
]
}binlog_server search_by_gtid_set
docker run --rm \
-v ~/binlog-server/config-gtid.json:/etc/binlog-server/config.json:ro \
-v ~/binlog-server/data-gtid:/var/lib/binlog-server/data:ro \
perconalab/percona-binlog-server:latest \
binlog_server search_by_gtid_set /etc/binlog-server/config.json \
7021a5ff-ac64-11f1-bc37-bc24112b899e:2117 | jq
{
"version": 1,
"status": "success",
"result": [
{
"name": "mysql-bin.000003",
"size": 169304257,
"uri": "file:///var/lib/binlog-server/data/mysql-bin.000003",
"previous_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:1-2116",
"added_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:2117",
"min_timestamp": "2026-09-09T17:32:51",
"max_timestamp": "2026-09-09T17:33:02"
}
]
}One file, instantly identified. What makes that possible is sitting right next to each binlog a json file. This is my favourite feature.
$ cat data-gtid/mysql-bin.000003.json | jq
{
"version": 1,
"size": 169304257,
"previous_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:1-2116",
"added_gtids": "7021a5ff-ac64-11f1-bc37-bc24112b899e:2117",
"min_timestamp": "2026-09-09T17:32:51",
"max_timestamp": "2026-09-09T17:33:02",
"last_sequence_number": 1
}binlog_server purge_binlogs
docker run --rm \
-v ~/binlog-server/config.json:/etc/binlog-server/config.json:ro \
-v ~/binlog-server/data:/var/lib/binlog-server/data \
perconalab/percona-binlog-server:latest \
binlog_server purge_binlogs /etc/binlog-server/config.json mysql-bin.000001
{"version":1,"status":"success","result":[{"name":"mysql-bin.000001","size":105825820,"uri":"file:///var/lib/binlog-server/data/mysql-bin.000001","min_timestamp":"2026-09-09T15:39:08","max_timestamp":"2026-09-09T17:32:14"}]}If we search the getid from binlog.000001, the error returned is
{"version":1,"status":"error","message":"The specified GTID set cannot be covered"}Conclusion
Overall, PBS (Percona Binlog Server) is a promising tool with some genuinely interesting features and functions. I am particularly impressed by the usability of search_by_timestamp, search_by_gtid_set. I’m looking forward to seeing where it goes as it heads toward a full server release.
If you’re working with MySQL binlogs and haven’t looked at PBS yet, I’d encourage you to give it a try and use it. Let me know if you need a hand in the setup or directly reach out to Percona via forum.
Kedar