35 Commits

Author SHA1 Message Date
Martin Kobetic
04a3e85793 Merge sorted pgids rather than resorting everything 2015-06-16 13:48:54 -06:00
sasha-s
bdc109bdc7 fix slice bounds out of range/maxAllocSize bugs
when accessing the node data we used to use cast to
*[maxAllocSize]byte, which breaks if we try to go across maxAllocSize boundary.

This leads to occasional panics.
Sample stacktrace:
```
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
github.com/boltdb/bolt.(*node).write(0xc208010f50, 0xc27452a000)
    $GOPATH/src/github.com/boltdb/bolt/node.go:228 +0x5a5
github.com/boltdb/bolt.(*node).spill(0xc208010f50, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/node.go:364 +0x506
github.com/boltdb/bolt.(*node).spill(0xc208010700, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/node.go:336 +0x12d
github.com/boltdb/bolt.(*node).spill(0xc208010620, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/node.go:336 +0x12d
github.com/boltdb/bolt.(*Bucket).spill(0xc22b6ae880, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/bucket.go:535 +0x1c4
github.com/boltdb/bolt.(*Bucket).spill(0xc22b6ae840, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/bucket.go:502 +0xac2
github.com/boltdb/bolt.(*Bucket).spill(0xc22f4e2018, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/bucket.go:502 +0xac2
github.com/boltdb/bolt.(*Tx).Commit(0xc22f4e2000, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/tx.go:150 +0x1ee
github.com/boltdb/bolt.(*DB).Update(0xc2080e4000, 0xc24d077508, 0x0, 0x0)
    $GOPATH/src/github.com/boltdb/bolt/db.go:483 +0x169
```
It usually happens when working with large (50M/100M) values.
One way to reproduce it is to change maxAllocSize in bolt_amd64.go to 70000 and run the tests.
TestBucket_Put_Large crashes.
2015-04-16 11:58:02 -07:00
Ben Johnson
349dacfd44 Update max alloc size on 386 arch. 2015-02-27 08:45:05 -07:00
Ben Johnson
b9c28b721a Increase max array size to 2GB.
This commit changes the maxAllocSize from 256GB to 2GB to handle large
values. It was previously 0xFFFFFFF and I tried adding one more "F" but
it caused an "array too large" error. I played around with the value
some more and found that 0x7FFFFFFF (2GB) is the highest allowed value.

This does not affect how the data is stored. It is simply used for type
converting pointers to array pointers in order to utilize zero copy
from the mmap.
2015-02-26 16:01:58 -07:00
Ben Johnson
515bf03ff9 Fix i686 build error. 2014-06-18 11:53:01 -06:00
Ben Johnson
fd7b0efeb5 Increase max nodes per page.
This commit changes the maxNodesPerPage constant so that it is essentially unlimited.
Previously, a single large transaction could create more nodes than the mock array
could handle.

Fixes #188.
2014-06-10 09:38:40 -06:00
Ben Johnson
12b36fe70c Fix freelist allocate(). 2014-05-19 14:11:32 -06:00
Ben Johnson
782ead0dbf Fix freelist allocation direction.
This commit fixes the freelist so that it frees from the beginning of the data file
instead of the end. It also adds a fast path for pages which can be allocated from
the first free pages and it includes read transaction stats.
2014-05-19 12:08:33 -06:00
Ben Johnson
698b07b074 Add nested buckets.
This commit adds the ability to create buckets inside of other buckets.
It also replaces the buckets page with a root bucket.

Fixes #56.
2014-04-11 12:36:54 -06:00
Ben Johnson
0e4d77d424 Add 'bolt pages'. 2014-03-21 22:34:54 -06:00
Ben Johnson
a1f43f4d60 Allow reads of unflushed nodes.
This commit allows cursors to read updated values from within the
RWTransaction.
2014-03-01 09:13:59 -07:00
Ben Johnson
8ad59edd02 API Documentation. 2014-02-13 10:58:27 -07:00
Ben Johnson
509e93dff4 Add freelist. 2014-02-10 14:04:01 -07:00
Ben Johnson
84939c21f6 Refactor node lookup. 2014-02-07 15:03:29 -07:00
Ben Johnson
0ed3dc3071 Rename sys ☞ buckets. 2014-02-05 22:15:47 -07:00
Ben Johnson
1a17a2cf1e Add RWTransaction.Put(). 2014-02-01 12:30:37 -05:00
Ben Johnson
26f6fefead Add RWTransaction.write(). 2014-01-30 22:20:50 -05:00
Ben Johnson
149d48fb9e Fix leaf/branch deserialization. 2014-01-30 00:11:46 -05:00
Ben Johnson
4ad445aa85 Add freelist page type. 2014-01-29 22:50:29 -05:00
Ben Johnson
4fc84daf2a Refactor meta.copy() and page.init(). 2014-01-29 22:35:58 -05:00
Ben Johnson
bb7a93f750 Add tpage.read() test. 2014-01-28 16:18:16 -05:00
Ben Johnson
ef590ecdfb lpage 2014-01-27 22:22:37 -05:00
Ben Johnson
192649f453 Intermediate. 2014-01-27 10:11:54 -05:00
Ben Johnson
1baa6d576a Initialize transaction/rwtransaction. 2014-01-26 15:29:06 -07:00
Ben Johnson
73ab1d420d TODO 2014-01-24 16:32:18 -07:00
Ben Johnson
20b26eac78 TODO 2014-01-24 12:51:56 -07:00
Ben Johnson
bce3e667df Intermediate commit. 2014-01-21 15:00:48 -07:00
Ben Johnson
153372abd4 Refactoring to RWCursor, RWTxn, and branch/leaf nodes and pages. 2014-01-17 15:23:39 -07:00
Ben Johnson
cec7b942e7 Add system buckets. 2014-01-14 13:01:02 -07:00
Ben Johnson
79d9b6bb5a Begin Transaction.Cursor(). 2014-01-13 10:35:04 -07:00
Ben Johnson
f1d7fe5b08 Finish open coverage. 2014-01-13 08:25:56 -07:00
Ben Johnson
ee24437bfc Initial db.open. 2014-01-11 22:51:01 -07:00
Ben Johnson
df8333328f DB.Open(), pages, and meta. 2014-01-10 07:32:12 -07:00
Ben Johnson
f922c1d2bc Move all C code into repo. 2014-01-09 09:07:10 -07:00
Ben Johnson
ebc9f0da9e Basic types. 2014-01-08 08:06:17 -07:00