1195 Commits

Author SHA1 Message Date
Benjamin Wang
01f29e9e9c
Merge pull request #817 from fuweid/13-backport-49eb212fa8ab67709ea460df01982504cf7fa4a1
[1.3] tests/robustness: switch to kill if no panic after 10sec
2024-08-09 12:40:20 +01:00
Wei Fu
99078466bb tests/robustness: switch to kill if no panic after 10sec
If file doesn't grow in 10 sec, bbolt won't trigger the following
errors:

* lackOfDiskSpace
* mapError
* resizeFileError
* unmapError

We should switch to kill instead of waiting for panic. In order to
trigger these errors, we should increase value size to 512.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit 49eb212fa8ab67709ea460df01982504cf7fa4a1)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-08-09 17:37:02 +08:00
Benjamin Wang
1b38fb35d5
Merge pull request #816 from fuweid/13-failpoint-backport
[1.3] backport failpoints(resizeFileError, lackOfDiskSpace) and dmflakey on XFS
2024-08-09 09:58:27 +01:00
Thomas Jungblut
70ab151d75 Add basic XFS powerfailure tests
This also introduces mkfs options, in case we need to accomodate for
non-default parameters here in the future.

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
(cherry picked from commit c27eedcf803fb5c6990d8846275c5136e06de1ae)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-08-09 09:39:33 +08:00
Wei Fu
0719297af8 tests: Update TestRestartFromPowerFailure
Update case with a combination of EXT4 filesystem's commit setting and
unexpected exit event. That EXT4 filesystem's commit is to sync all its data
and metadata every seconds. The kernel can help us sync even if that
process has been killed. With different commit setting, we can simulate
that case that kernel syncs half part of dirty pages before power
failure. And for unexpected exit event, we can kill that process
randomly or panic at failpoint instead of fixed code path.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit 4c3a80b2c06cb5a39eeabe422f718a1b6cfc7be2)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-08-09 09:34:35 +08:00
Marcondes Viana
79bcca0e51 tests: add failpoint to simulate lack of disk space
Signed-off-by: Marcondes Viana <marju10@gmail.com>
(cherry picked from commit 5ddbd0c94e221668fff3f9d234bd6fbb74d9457e)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-08-09 09:31:00 +08:00
Benjamin Wang
ec68f060a1 add failpoint 'resizeFileError' to simulate file.Truncate error
Signed-off-by: Benjamin Wang <wachao@vmware.com>
(cherry picked from commit 465077b9e2f3d0bfe593beda5a4b04ea1cd0915a)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-08-09 09:27:07 +08:00
Benjamin Wang
78b80f5c6d
Merge pull request #813 from henrybear327/release-1.3
[release-1.3] go version bump from 1.21.12 to 1.21.13
2024-08-08 22:05:57 +01:00
Chun-Hung Tseng
2e1187ba48
[release-1.3] go version bump from 1.21.12 to 1.21.13
Reference: https://github.com/etcd-io/etcd/issues/18419
Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-08-08 21:55:25 +02:00
Fu Wei
5292f71eda
[1.3] backport go-dmflakey and robustness test (#812)
* tx.go: introduce failpoint before fdatasync

It can be used for power failure cases.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit 445a07b6cb837d1196f9fceb7655900031ca093f)
Signed-off-by: Wei Fu <fuweid89@gmail.com>

* tests/*: introduce go-dmflakey

This commit is to add go-binding for go-dmflakey. It's used to simulate
powerfailure with common filesystems.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit b05bec2fe3f4de1650daf868e507e686d86770a5)
Signed-off-by: Wei Fu <fuweid89@gmail.com>

* tests/robustness: init with powerfailure case

Add `Robustness Test` pipeline for robustness test cases.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit 300e72ab8a0d3b730dbafb3b5463f00615bbfc9b)
Signed-off-by: Wei Fu <fuweid89@gmail.com>

---------

Signed-off-by: Wei Fu <fuweid89@gmail.com>
2024-08-08 16:22:18 +01:00
Benjamin Wang
3c128a606d
Merge pull request #805 from tjungblu/stdout
[1.3] cmd/bbolt: write bench results to stdout
2024-08-06 07:20:54 +01:00
Thomas Jungblut
c13ec0a16d cmd/bbolt: write bench results to stdout
backport of #767 to fix the missing output on nightly benchmarks

Co-authored-by: Iván Valdés Castillo <iv@nvald.es>
Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2024-08-05 11:26:48 +02:00
Benjamin Wang
14c8aaf575
Merge pull request #802 from tjungblu/nighty_bench_3.5
[1.3] backport make build target and go bench output
2024-08-01 08:23:35 +01:00
Thomas Jungblut
138cbb3bc1 ensure build and bench output in 3.5
manual backport of #765 and partially #664, to make the nightly
benchmark work

Signed-off-by: Thomas Jungblut <tjungblu@redhat.com>
2024-07-31 13:28:14 +02:00
Benjamin Wang
71159e32bf
Merge pull request #785 from thedtripp/release-1.3
[1.3] Bump Go version to 1.21.12: GO-2024-2963 fix
2024-07-04 12:53:48 +01:00
D Tripp
616a4e1da3 Bump Go version to 1.21.12: CVE 2024-2963 fix
Signed-off-by: D Tripp <38776199+thedtripp@users.noreply.github.com>
2024-07-04 01:09:50 +00:00
Benjamin Wang
f9936edf0f
Merge pull request #763 from ArkaSaha30/release-1.3-cve-2024-24790
[1.3]Bump Go version to 1.21.11: CVE 2024-24790 fix
2024-06-05 16:22:56 +01:00
ArkaSaha30
7b71896470
Bump Go version to 1.21.11: CVE 2024-24790 fix
Signed-off-by: ArkaSaha30 <arkasaha30@gmail.com>
2024-06-05 15:15:22 +05:30
Benjamin Wang
96c3f52f23
Merge pull request #754 from lavishpal/update/go-to-1.21.10
[1.3] dependency : update the go version to 1.21.10
2024-05-10 18:03:19 +01:00
Lavish pal
f5a30b77af update the go version to 1.21.10
Signed-off-by: Lavish pal <lvishpal408@gmail.com>
2024-05-10 20:03:16 +05:30
Benjamin Wang
014b0285cc
Merge pull request #744 from ahrtr/cursor_20240502
[1.3] Ensure a cursor can continue to iterate elements in reverse direction by call Next when it has already reached the beginning
v1.3.10
2024-05-03 06:49:59 +01:00
Benjamin Wang
2d48e1df03 Ensure a cursor can continue to iterate elements in reverse direction by call Next when it has already reached the beginning
Signed-off-by: Benjamin Wang <benjamin.ahrtr@gmail.com>
2024-05-02 16:43:50 +01:00
Benjamin Wang
886753c41a
Merge pull request #717 from henrybear327/fix/remove_UnsafeSlice
Remove UnsafeSlice and use unsafe.Slice
2024-04-07 19:02:39 +01:00
Chun-Hung Tseng
9e3d732b27
Remove UnsafeSlice and use unsafe.Slice
References:
- https://github.com/etcd-io/bbolt/pull/713
- https://github.com/etcd-io/bbolt/pull/566

Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-04-06 21:40:37 +02:00
Benjamin Wang
2c29534715
Merge pull request #713 from henrybear327/cve/CVE-2023-45288-release-1.3
Bump go toolchain version to address CVE-2023-45288 for release-1.3
2024-04-05 19:24:09 +01:00
Chun-Hung Tseng
9b60c134ed
Address linter issues
Changes (align with the main branch):
- remove rand.Seed(s)
- use rand.Read from "crypto/rand"
- add //nolint:all for (*reflect.SliceHeader)(slice) -> will fix in a 
follow-up PR

Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
Co-authored-by: Iván Valdés Castillo <iv@nvald.es>
2024-04-05 14:22:02 +02:00
Chun-Hung Tseng
0b59bc1317
Bump go toolchain version to address CVE-2023-45288
Changes:
- Bump toolchain version to 1.21.9 due to CVE-2023-45288
- run `go mod tidy`

Reference:
- PR etcd #17703

Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-04-05 14:21:41 +02:00
Benjamin Wang
9f3524b608
Merge pull request #716 from henrybear327/feat/introduce-go-version-file
Adopt the .go-version approach for toolchain versioning
2024-04-05 13:16:35 +01:00
Chun-Hung Tseng
67a267830e
Adopt the .go-version approach for toolchain versioning
Signed-off-by: Chun-Hung Tseng <henrybear327@gmail.com>
2024-04-05 11:53:07 +02:00
Benjamin Wang
50aef2646b
Merge pull request #683 from Elbehery/backport_fix_cli_panics
[1.3] Fix `bbolt keys` and `bbolt get` to prevent them from panicking when no parameter provided
v1.3.9
2024-02-01 13:56:13 +00:00
Mustafa Elbehery
25dacb34be fix cli cmds panics
Signed-off-by: Mustafa Elbehery <melbeher@redhat.com>
2024-01-31 19:23:36 +01:00
Benjamin Wang
e102fcf8a2
Merge pull request #639 from fuweid/cp-copy-key-before-seek
[1.3] bucket.Put: copy key before seek
2023-12-18 16:53:49 +00:00
Wei Fu
fabe2fb55e *: copy key before comparing during CreateBucket
It's follow-up of #637.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit 62d80260de277168b2d59779bfb03c6ebfda08f4)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-12-18 23:39:03 +08:00
Wei Fu
50ddad0f00 bucket: copy key before Put
Application might change key value after seeking and before real put.
This unexpected behaviour could corrupt database. When users file issue,
maintainers doesn't know application behaviour. It could be caused by
data race. This patch is to prevent such case and save maintainers' time.

Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit a05ec68aaafcf77e22b9da83bd4069cad8cba39d)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-12-18 23:34:58 +08:00
Wei Fu
b3bdd17686 tests/robustness: add issue72 reproducer
Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit 1b080787075bb3ec06a414c754da9ef66bda4071)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-12-18 23:33:23 +08:00
Wei Fu
d520aaac7c *: introduce failpoint beforeBucketPut
Signed-off-by: Wei Fu <fuweid89@gmail.com>
(cherry picked from commit 324df9cd264b7f2c4504ea61758a4ea89373e4ce)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-12-18 23:31:06 +08:00
Evgenii Stratonikov
0a521c0b48 bucket: allow to allocate key on stack in Put()
As per `go build -gcflags -m ./... 2>&1`:

Old behaviour:
```
./bucket.go:148:31: leaking param: key
./bucket.go:192:42: leaking param: key
./bucket.go:271:22: leaking param: key
```

Now:
```
./bucket.go:148:31: key does not escape
./bucket.go:192:42: key does not escape
./bucket.go:271:22: key does not escape
```

Signed-off-by: Evgenii Stratonikov <fyfyrchik@runbox.com>
(cherry picked from commit 71a59caf31ca42c3deb8967ceee0a4804d220bf4)
Signed-off-by: Wei Fu <fuweid89@gmail.com>
2023-12-14 08:34:43 +08:00
Marek Siarkowicz
defa564feb
Merge pull request #600 from Richabanker/release-1.3
Backport concurrent_test.go to release-1.3 branch
2023-11-08 05:58:11 +01:00
Richa Banker
fe23f523fd Backport concurrent_test.go to release-1.3 branch
Signed-off-by: Richa Banker <richabanker@google.com>
2023-11-07 15:43:19 -08:00
Benjamin Wang
42a914d8be
Merge pull request #586 from ahrtr/1.3_64bit_align_20231025
[1.3] Ensure the stats is always 64bit aligned
v1.3.8
2023-10-25 20:10:10 +01:00
Benjamin Wang
f9d290fdfd ensure the stats is always 64bit aligned
The first word in an allocated struct can be relied upon to be
64-bit aligned. Refer to https://pkg.go.dev/sync/atomic#pkg-note-BUG.

Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-10-25 16:19:38 +01:00
Benjamin Wang
4a1773274a
Merge pull request #444 from jmhbnz/backport-failpoints-injection-2
[1.3] Backport perform unmap when mlock fails or both meta pages corrupted
2023-03-31 06:20:08 +08:00
James Blair
95acc50921
Backport add test cases to simulate mlock failure.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-30 21:54:14 +13:00
James Blair
7a13798511
Backport perform unmap when failing to mlock or both meta pages corrupted.
Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-30 21:54:05 +13:00
Benjamin Wang
ad36005fa6
Merge pull request #439 from jmhbnz/backport-failpoints-injection
[1.3] Backport change to error handling logic
2023-03-29 06:40:49 +08:00
James Blair
8165a40522
Backport change to error handling logic in db.close().
Signed-off-by: James Blair <mail@jamesblair.net>
2023-03-26 11:50:05 +13:00
Benjamin Wang
e4e06d2245 update the usage of surgery command
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-02-18 10:36:42 +08:00
Josh Rickmar
110891517d Avoid syscall.Syscall use on OpenBSD
Syscall numbers are not stable on OpenBSD, and hardcoding the msync
syscall number will break bbolt on future versions of OpenBSD.  Use
the libc wrapper provided by golang.org/x/sys/unix instead.

Signed-off-by: Josh Rickmar <jrick@zettaport.com>
2023-02-16 10:25:51 +08:00
Benjamin Wang
da2f2a53f6 remove changelog from release-1.3
Signed-off-by: Benjamin Wang <wachao@vmware.com>
v1.3.7
2023-01-31 05:21:49 +08:00
Benjamin Wang
fc361799ce added summary on the changes in v1.3.7
Signed-off-by: Benjamin Wang <wachao@vmware.com>
2023-01-29 20:51:54 +08:00