* Fix testDB_Close_PendingTx to do something with the writable arg and stop it from closing twice
* Fix Close() to wait for view transactions by getting a full lock on mmaplock
* Fix the TestTx_Check_ReadOnly to close the view transaction
* Fix the TestTx_Commit_ErrTxNotWritable to close the view transaction
Fixes an issue where failing to open a non-existent database in ReadOnly
mode would make you unable to properly initialize it in ReadWrite mode
afterwards due to a hanging lock.
This allows enabling NoSync when you only have access to the Options
passed into Open() and not the returned DB (as is frequently the case
with libraries).
When the database has a lot of freepages, the cost to sync all
freepages down to disk is high. If the total database size is
small (<10GB), and the application can tolerate ~10 seconds
recovery time, then it is reasonable to simply not sync freelist
and rescan the db to rebuild freelist on recovery.
Read txns would lock pages allocated after the txn, keeping those pages
off the free list until closing the read txn. Instead, track allocating
txid to compute page lifetime, freeing pages if all txns between
page allocation and page free are closed.
The existing append-based implementation left a hanging reference to
the last tx.
For example, if db.txs was:
[]*Tx{0x1, 0x2, 0x3, 0x4, 0x5}
and we removed the second element, db.txs would now be:
[]*Tx{0x1, 0x3, 0x4, 0x5, 0x5}[:4]
The garbage collector cannot reclaim anything anywhere in a slice,
even pointers between its len and cap, because the len can always
be extended up to the cap.
This hanging reference to the Tx could last indefinitely,
and since the Tx has a reference to user-provided functions,
which could be closures, this bug could prevent arbitrary
amounts of user garbage from being collected.
Since db.txs is unordered anyway, switch to a simpler--and O(1) instead
of O(n)--implementation. Swap the last element into the spot to be
deleted, nil out the original last element, and shrink the slice.
The subtraction for `TxN` was previously transposed which caused
the result to be a negative number. This change alters the order
to return the correct (positive) result.
Bolt stores the two latest transactions' metadata, but previously did
not recover from validation failures in the latest by using the second
latest. Fix this by correctly handling validation failures in db.go, as
well as returning the metadata with highest txid which is also valid in
DB.meta().
Signed-off-by: Aleksa Sarai <asarai@suse.de>
This commit refactors the test suite to make it cleaner and to use the
standard testing library better. The `assert()`, `equals()`, and `ok()`
functions have been removed and some test names have been changed for
clarity.
No functionality has been changed.
InitialMmapSize is the initial mmap size of the database in bytes.
Read transaction won't block write transaction if InitialMmapSize
is large enough to handle mmap size.
Copied from https://github.com/boltdb/bolt/pull/432.
- ErrInvalid is returned when a data file is not a Bolt-formatted
database.
- ErrVersionMismatch is returned when the data file was created with a
different version of Bolt.
- ErrChecksum is returned when either meta page checksum does not match.
Also:
- Do not wrap errors from os.Stat, so that a caller could handle os.Stat
errors just like it can handle errors from os.Open that bolt.Open
might return.
- Name tests consistently, following the pattern "TestOpen_*".
- Remove deferred calls to `os.Remove(path)`.
The calls are not only unnecessary, but also in all cases `os.Remove`
returns an error that is ignored. All those calls are meant to remove
a file that was already removed by `tmpfile()`.
- Combine "bad path" tests and use filepath.Join to build the path.
This adds MmapFlags to DB.Options in case we need syscall.MAP_POPULATE
flag in Linux 2.6.23+ to do the sequential read-ahead, as discussed in [1].
---
[1]: https://github.com/coreos/etcd/issues/3786
Only grow the database size when the high watermark increases.
We also grows the database size a little bit aggressively to
save a few ftruncates.
I have tested this on various environments. The performance impact
is ignorable with 16MB over allocation. Without over allocation,
the performance might decrease 100% when each Tx.Commit needs a new
page on a very slow disk (seek time dominates the total write).
This commit adds documentation to clarify that read-only and read-write
transactions should not be mixed in the same goroutine as it can cause
deadlocks during remapping.
See: https://github.com/boltdb/bolt/issues/378
This commit adjusts the minimum mmap size from 1MB to 32KB. The
previous limit was arbitrary and causes wasted space for very small
databases.
Thanks to @mcuadros for submitting the original pull request:
https://github.com/boltdb/bolt/pull/351
John Howard
Gyuho Lee
Paul
Gyuho Lee
CJ DiMaggio
Rodrigo Coelho
Tommi Virtanen
Anthony Romano
lorneli
Raphael Geronimi
Tyler Treat
Sokolov Yura aka funny_falcon
Xiang Li
Josh Bleecher Snyder
Ben Johnson
Aleksa Sarai
Chris Hines
Alexander Morozov
Rodolfo Carvalho
sasha-s