mirror of
https://gitflic.ru/project/erthink/libmdbx.git
synced 2025-01-27 18:39:24 +00:00
mdbx-doc: доработка/актуализация раздела "Restrictions & Caveats" (backport).
This commit is contained in:
parent
03685aba5a
commit
c615e4d0a6
@ -42,18 +42,33 @@ long-lived read transactions, and using the \ref MDBX_LIFORECLAIM mode
|
|||||||
which addresses subsequent performance degradation. The "next" version
|
which addresses subsequent performance degradation. The "next" version
|
||||||
of libmdbx (aka \ref MithrilDB) will completely solve this.
|
of libmdbx (aka \ref MithrilDB) will completely solve this.
|
||||||
|
|
||||||
- Avoid suspending a process with active transactions. These would then be
|
Nonetheless, situations that encourage lengthy read transactions while
|
||||||
"long-lived" as above.
|
intensively updating data should be avoided. For example, you should
|
||||||
|
avoid suspending/blocking processes/threads performing read
|
||||||
|
transactions, including during debugging, and use transaction parking if
|
||||||
|
necessary.
|
||||||
|
|
||||||
- Avoid aborting a process with an active read-only transaction in scenarios
|
You should also beware of aborting processes that perform reading
|
||||||
with high rate of write transactions. The transaction becomes "long-lived"
|
transactions. Despite the fact that libmdbx automatically checks and
|
||||||
as above until a check for stale readers is performed or the LCK-file is
|
cleans readers, as an a process aborting (especially with core dump) can
|
||||||
reset, since the process may not remove it from the lockfile. This does
|
take a long time, and checking readers cannot be performed too often due
|
||||||
not apply to write transactions if the system clears stale writers, see
|
to performance degradation.
|
||||||
above.
|
|
||||||
|
This issue will be addressed in MithrlDB and one of libmdbx releases,
|
||||||
|
presumably in 2025. To do this, nonlinear GC recycling will be
|
||||||
|
implemented, without stopping garbage recycling on the old MVCC snapshot
|
||||||
|
used by a long read transaction.
|
||||||
|
|
||||||
|
After the planned implementation, any long-term reading transaction will
|
||||||
|
still keep the used MVCC-snapshot (all the database pages forming it)
|
||||||
|
from being recycled, but it will allow all unused MVCC snapshots to be
|
||||||
|
recycled, both before and after the readable one. This will eliminate
|
||||||
|
one of the main architectural flaws inherited from LMDB and caused the
|
||||||
|
growth of a database in proportion to a volume of data changes made
|
||||||
|
concurrently with a long-running read transaction.
|
||||||
|
|
||||||
|
|
||||||
## Large data items and huge transactions
|
## Large data items
|
||||||
|
|
||||||
MDBX allows you to store values up to 1 gigabyte in size, but this is
|
MDBX allows you to store values up to 1 gigabyte in size, but this is
|
||||||
not the main functionality for a key-value storage, but an additional
|
not the main functionality for a key-value storage, but an additional
|
||||||
@ -71,6 +86,18 @@ and in the absence of a sufficient sequence of free pages, increase the
|
|||||||
DB file. Thus, for long values, MDBX provides maximum read performance
|
DB file. Thus, for long values, MDBX provides maximum read performance
|
||||||
at the expense of write performance.
|
at the expense of write performance.
|
||||||
|
|
||||||
|
Some aspects related to GC have been refined and improved in 2022 within
|
||||||
|
the first releases of the 0.12.x series. In particular the search for
|
||||||
|
free consecutive/adjacent pages through GC has been significantly
|
||||||
|
speeded, including acceleration using NOEN/SSE2/AVX2/AVX512
|
||||||
|
instructions.
|
||||||
|
|
||||||
|
This issue will be addressed in MithrlDB and refined within one of
|
||||||
|
0.15.x libmdbx releases, presumably at end of 2025.
|
||||||
|
|
||||||
|
|
||||||
|
### Huge transactions
|
||||||
|
|
||||||
A similar situation can be with huge transactions, in which a lot of
|
A similar situation can be with huge transactions, in which a lot of
|
||||||
database pages are retired. The retired pages should be put into GC as a
|
database pages are retired. The retired pages should be put into GC as a
|
||||||
list of page numbers for future reuse. But in huge transactions, such a
|
list of page numbers for future reuse. But in huge transactions, such a
|
||||||
@ -80,20 +107,15 @@ you delete large amounts of information from the database in a single
|
|||||||
transaction, MDBX may need to increase the database file to save the
|
transaction, MDBX may need to increase the database file to save the
|
||||||
list of pages to be retired.
|
list of pages to be retired.
|
||||||
|
|
||||||
Both of these issues will be addressed in MithrilDB.
|
This issue was fixed in 2022 within the first releases of the 0.12.x
|
||||||
|
series by `Big Foot` feature, which now is enabled by default.
|
||||||
|
See \ref MDBX_ENABLE_BIGFOOT build-time option.
|
||||||
|
|
||||||
#### Since v0.12.1 and later
|
The `Big Foot` feature which significantly reduces GC overhead for
|
||||||
Some aspects related to GC have been refined and improved, in particular:
|
processing large lists of retired pages from huge transactions. Now
|
||||||
|
libmdbx avoid creating large chunks of PNLs (page number lists) which
|
||||||
- The search for free consecutive/adjacent pages through GC has been
|
required a long sequences of free pages, aka large/overflow pages. Thus
|
||||||
significantly speeded, including acceleration using
|
avoiding searching, allocating and storing such sequences inside GC.
|
||||||
NOEN/SSE2/AVX2/AVX512 instructions.
|
|
||||||
|
|
||||||
- The `Big Foot` feature which significantly reduces GC overhead for
|
|
||||||
processing large lists of retired pages from huge transactions. Now
|
|
||||||
libmdbx avoid creating large chunks of PNLs (page number lists) which
|
|
||||||
required a long sequences of free pages, aka large/overflow pages. Thus
|
|
||||||
avoiding searching, allocating and storing such sequences inside GC.
|
|
||||||
|
|
||||||
|
|
||||||
## Space reservation
|
## Space reservation
|
||||||
@ -135,9 +157,11 @@ On the other hand, MDBX allow calling \ref mdbx_env_close() in such cases to
|
|||||||
release resources, but no more and in general this is a wrong way.
|
release resources, but no more and in general this is a wrong way.
|
||||||
|
|
||||||
#### Since v0.13.1 and later
|
#### Since v0.13.1 and later
|
||||||
Начиная с версии 0.13.1 в API доступна функция \ref mdbx_env_resurrect_after_fork(),
|
|
||||||
которая позволяет пере-использовать в дочерних процессах уже открытую среду БД,
|
Starting from the v0.13.1 release, the \ref mdbx_env_resurrect_after_work()
|
||||||
но строго без наследования транзакций от родительского процесса.
|
is available, which allows you to reuse an already open database
|
||||||
|
environment in child processes, but strictly without inheriting any
|
||||||
|
transactions from a parent process.
|
||||||
|
|
||||||
|
|
||||||
## Read-only mode
|
## Read-only mode
|
||||||
@ -149,9 +173,6 @@ to without-LCK mode on appropriate errors (`EROFS`, `EACCESS`, `EPERM`)
|
|||||||
if the read-only mode was requested by the \ref MDBX_RDONLY flag which is
|
if the read-only mode was requested by the \ref MDBX_RDONLY flag which is
|
||||||
described below.
|
described below.
|
||||||
|
|
||||||
The "next" version of libmdbx (\ref MithrilDB) will solve this issue for the "many
|
|
||||||
readers without writer" case.
|
|
||||||
|
|
||||||
|
|
||||||
## Troubleshooting the LCK-file
|
## Troubleshooting the LCK-file
|
||||||
1. A broken LCK-file can cause sync issues, including appearance of
|
1. A broken LCK-file can cause sync issues, including appearance of
|
||||||
@ -166,8 +187,6 @@ readers without writer" case.
|
|||||||
multiple processes in a lock-free manner and any locking is
|
multiple processes in a lock-free manner and any locking is
|
||||||
unwise due to a large overhead.
|
unwise due to a large overhead.
|
||||||
|
|
||||||
The "next" version of libmdbx (\ref MithrilDB) will solve this issue.
|
|
||||||
|
|
||||||
\note Workaround: Just make all programs using the database close it;
|
\note Workaround: Just make all programs using the database close it;
|
||||||
the LCK-file is always reset on first open.
|
the LCK-file is always reset on first open.
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user