Percona Monthly Bug Report: August 2020

Percona Monthly Bug Report August 2020

Percona Monthly Bug Report August 2020At Percona, we operate on the premise that full-transparency makes a product better. We strive to build the best open-source database products, but also to help you manage any issues that arise in any of the databases that we support. And, in true open-source form, report back on any issues or bugs you might encounter along the way.

We constantly update our bug reports and monitor other boards to ensure we have the latest information, but we wanted to make it a little easier for you to keep track of the most critical ones. This monthly post is a central place to get information on the most noteworthy open and recently resolved bugs. 

In this edition of our monthly bug report, we have the following list of bugs:

Percona Server/MySQL Bugs

PS-7264(MySQL#83799):  [ERROR] InnoDB: dict_load_foreigns() returned 38 for ALTER TABLE

Affects Version/s: 5.6,5.7  [Tested/Reported version 5.6.49,5.7.30 ]

Critical since a user can easily hit this bug when performing ALTER on the table with FK for specified cases in the bug report. With debug build, it crashes. 

MySQL#91977: Dropping Large Table Causes Semaphore Waits; No Other Work Possible

Affects Version/s: 5.7,8.0  [Tested/Reported version 5.7.23, 8.0.12]

Running DROP/ALTER on the large table could lead to this bug. DROP/ALTER query will be stuck in ‘checking permissions’ state  and later it may crash mysqld due to long semaphore

Wait. It’s critical since it can result in unplanned downtime. The issue also evident with the pt-online-schema-change tool while performing ALTER operations on large tables.

PS-6990(MySQL#100118): Server doesn’t restart because of too many gaps in the mysql.gtid_executed table

Affects Version/s: 8.0  [Tested/Reported version 8.0.19, 8.0.20,8.0.21]

Fixed Version/s: 8.0.21 (Next release)

This issue introduces due to the Clone plugin replication coordinates worklog (WL#9211) changes for updating the mysql.gtid_executed table. With this bug, Replica not usable, as mysql.gtid_executed is getting huge, crashes, and never comes back because of the way how the GTID compression works.

PS-7203:  audit plugin memory leak on replicas when opening tables.

Affects Version/s: 5.7,8.0  [Tested/Reported version 5.7.26-29, 5.7.29-32]

Fixed Version/s: 5.7.31, 8.0.21 (Next release)

Audit plugin could leak memory resulting in OOM on the slave if STATEMENT base replication is used.

PS-7163(MySQL#99286):Concurrent update cause crash in row_search_mvcc

Affects Version/s: 5.7  [Tested/Reported version 5.7.29]

Concurrent updates on the same record (DELETE/INSERT and UPDATE from different sessions) could lead to this crash. This bug is critical since such operations are common for many database environments.

Percona Xtradb Cluster

PXC-3373: [ERROR] WSREP: Certification exception: Unsupported key prefix: ^B: 71 (Protocol error) at galera/src/key_set.cpp:throw_bad_prefix():152

Affects Version/s: 5.7   [Tested/Reported 5.7.30]

IST to the lower version node will fail with this error when the write load is running on the Donor node. Rolling upgrade is the most commonly used method resulting in a different version of pxc nodes for while and in such cases, the user can experience this issue easily.

Possible workaround,

  • Upgrade pxc node to have the same version across the cluster.
  • Stop write load on the donor node while IST is running. 

PXC-3352: Unexpected ERROR 1205 modifying a child table in an FK relationship

Affects Version/s: 5.7   [Tested/Reported 5.7.28]

Fixed Version/s: 5.7.31, 8.0.20 (Next release)

Affected users using foreign keys with PXC, When deleting/updating from a child table in a FK relationship if the parent table’s referenced rows are locked, the operation on a child table failed with lock wait timeout error when the parent table is unlocked.  

Percona Xtrabackup

PXB-2237: PXB crashes during a backup when an encrypted table is updated

Affects Version/s:  2.4  [Tested/Reported version 2.4.20]

Databases with encrypted tables are affected by this bug. As a workaround, Taking backup in non-peak hours could avoid this crash.

PXB-2178: Restoring datadir from partial backup results in an inconsistent data dictionary

 Affects Version/s:  8.0  [Tested/Reported version 8.0.11

As a result of this bug after restored, you will see additional database/s which were not part of a partial backup. Issue evident only in Xtrabackup 8.0 due to new data dictionary implementation in MySQL 8.0 version, this issue is not reproducible with xtrabackup 2.4. 

The workaround for this issue is to use “DROP DATABASE IF EXISTS” for dropping unwanted extra database/s. 

Percona Toolkit

PT-1747: pt-online-schema-change was bringing the database into a broken state when applying the “rebuild_constraints” foreign keys modification method if any of the child tables were blocked by the metadata lock.

Affects Version/s:  3.x   [Tested/Reported version 3.0.13]

Critical bug since it can cause data inconsistency in the user environment. It potentially affects who rebuilds tables with foreign keys.

PT-1853: pt-online-schema-change doesn’t handle self-referencing foreign keys properly

When using pt-osc to change a table that has a self FK pt-osc creates the FK pointing to the old table instead of pointing to the _new table. Because of it pt-osc needs to rebuild the FK after swapping the tables (DROP the FK and recreating it again pointing to the _new table). This can cause issues because the INPLACE algorithm is supported when foreign_key_checks is disabled. Otherwise, only the COPY algorithm is supported.

Affects Version/s:  3.x  [Tested/Reported version 3.2.0]

Fixed Version/s: 3.2.1

Affects who rebuild tables with a self-referencing foreign key.

PMM  [Percona Monitoring and Management]

PMM-4547: MongoDB dashboard replication lag count incorrect

Affects Version/s:  2.9  [Tested/Reported version 2.0,2.9.0]

Replica lag takes the lag value from the delayed node which makes it unreliable for all nodes. So PMM cannot be used to monitor delays.

PMM-6336:   Incompatible pmm-admin options: ‘–disable-queryexamples’ and ‘–query-source=perfschema’

In case the user doesn’t want to show a query example that has “Sensitive Data” on PMM Query Analytics UI  they can disable it with the help of –disable-queryexamples option in pmm-admin.

But this option is not working as expected when mysql instance added using –query-source perfschema option and on PMM Query Analytics UI  query examples is visible with original data.

Workaround: we do not see this issue when using –query-source slowlog 

Affects Version/s:  2.9  [Tested/Reported version 2.9]

Fixed version/s: 2.10.0

PMM-5823: pmm-server log download and api to get the version failed with timeout

Affects Version/s:  2.8  [Tested/Reported version 2.2]

Occurring at irregular intervals and only affected to pmm docker installation with no external internet access from pmm-server docker container. The issue only visible for a while(around 5-10 mins) after starting pmm-server later you will not see this issue.


We welcome community input and feedback on all our products. If you find a bug or would like to suggest an improvement or a feature, learn how in our post, How to Report Bugs, Improvements, New Feature Requests for Percona Products.

For the most up-to-date information, be sure to follow us on Twitter, LinkedIn, and Facebook.

Quick References:

Percona JIRA  

MySQL Bug Report

Report a Bug in a Percona Product


About Percona:

As the only provider of distributions for all three of the most popular open source databases—PostgreSQL, MySQL, and MongoDB—Percona provides expertise, software, support, and services no matter the technology.

Whether its enabling developers or DBAs to realize value faster with tools, advice, and guidance, or making sure applications can scale and handle peak loads, Percona is here to help.

Percona is committed to being open source and preventing vendor lock-in. Percona contributes all changes to the upstream community for possible inclusion in future product releases.

by Lalit Choudhary via Percona Database Performance Blog