8 months ago_just occasionally_ we may under-read some bytes, giving us less than 32 bytes for a key. master
B Tasker []
_just occasionally_ we may under-read some bytes, giving us less than 32 bytes for a key.

ChaCha will throw an exception if this happens, so test the key and skip over any that are too short. This _may_ lead to us queuing a shorter block

8 months agoLink to the right page...
B Tasker []
Link to the right page...

8 months agoAdd some links.
B Tasker []
Add some links.

The article will be up soon

8 months agoUpdate README
B Tasker []
Update README

8 months agoBugfix - without this fix, following mutate, our key will be 32 bytes of 0s
B Tasker []
Bugfix - without this fix, following mutate, our key will be 32 bytes of 0s

8 months agoPrint a big old warning if we're doing something insecure.
B Tasker []
Print a big old warning if we're doing something insecure.

This except only exists because I've been doing quite a bit of testing on a Pi which doesn't have RDRAND

8 months agoActually, we should probably mix in RDRAND _before_ passing off to ChaCha
B Tasker []
Actually, we should probably mix in RDRAND _before_ passing off to ChaCha

That way, even if either stream is biased, ChaCha should randomise it for us.

Doing it the other way round means that if you were able to correctly predict what we should be getting from RDRAND you could reverse it back out (though its not clear what you'd gain from doing so, it still feels like a hole)

8 months agoDon't push the final block to output.
B Tasker []
Don't push the final block to output.

It's used as the next input, so we shouldn't really be outputting it. That way numbers used in key material *and* input are never output.

ChaCha20 isn't/shouldn't be susceptible to known-plaintext, so this is probably irrelevant, but seems better to err on the side of caution

8 months agoOops, https not http
B Tasker []
Oops, https not http

8 months agoUpdate README
B Tasker []
Update README

8 months agoAdd license to top of script - BSD 3 Clause
B Tasker []
Add license to top of script - BSD 3 Clause

8 months agoAdd a README
B Tasker []
Add a README

8 months agoQuick tidy up of commented out stuff/whitespace
B Tasker []
Quick tidy up of commented out stuff/whitespace

It's not exactly lightning fast

pi@nextcloudpi:~ $ dd if=/tmp/csprng of=foo count=20K
17928+2552 records in
19440+0 records out
9953280 bytes (10 MB, 9.5 MiB) copied, 32.632 s, 305 kB/s

But ent seems happy enough with it

pi@nextcloudpi:~ $ cat foo | ent
Entropy = 7.999982 bits per byte.

Optimum compression would reduce the size
of this 9953280 byte file by 0 percent.

Chi square distribution for 9953280 samples is 242.99, and randomly
would exceed this value 69.53 percent of the times.

Arithmetic mean value of data bytes is 127.4848 (127.5 = random).
Monte Carlo value for Pi is 3.141437596 (error 0.00 percent).
Serial correlation coefficient is 0.000116 (totally uncorrelated = 0.0).

Obviously [*any* assessment is a bit finger in the air](https://www.bentasker.co.uk/documentation/security/287-understanding-the-difficulty-of-assessing-entropy)

pi@nextcloudpi:~ $ cat foo | rngtest
rngtest 2-unofficial-mt.14
Copyright (c) 2004 by Henrique de Moraes Holschuh
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

rngtest: starting FIPS tests...
rngtest: entropy source exhausted!
rngtest: bits received from input: 79626240
rngtest: FIPS 140-2 successes: 3979
rngtest: FIPS 140-2 failures: 2
rngtest: FIPS 140-2(2001-10-10) Monobit: 0
rngtest: FIPS 140-2(2001-10-10) Poker: 1
rngtest: FIPS 140-2(2001-10-10) Runs: 0
rngtest: FIPS 140-2(2001-10-10) Long run: 1
rngtest: FIPS 140-2(2001-10-10) Continuous run: 0
rngtest: input channel speed: (min=3.267; avg=1202.343; max=9536.743)Mibits/s
rngtest: FIPS tests speed: (min=7.033; avg=15.940; max=16.629)Mibits/s
rngtest: Program run time: 4832750 microseconds

8 months agoOops. didn't mean to leave that there
B Tasker []
Oops. didn't mean to leave that there

8 months agoUse spare data from the key-generation to mutate the key part way through iterations
B Tasker []
Use spare data from the key-generation to mutate the key part way through iterations

The aim of this is to provide some additional protection against back-tracking.

That's in _theory_ of course. In practice, I'm using a GC'd language, so there's no way to be sure the previous keys are in memory just waiting to be plucked out,

But, ignoring that, an adversary that can see current state and current output _should_ be unable to backtrack more than 24 values (24 being 1/2 an iteration loop), because the key will have been mutated based on a fragment of much, much earlier data.

The other possibility, of course, is that what we're actually doing here is keeping data in memory that could be used to re-construct values *before* the last full-key change. That would be bad

8 months agoUse the full seed as the input to chacha rather than the first 32 bytes onwards
B Tasker []
Use the full seed as the input to chacha rather than the first 32 bytes onwards

Although it doesn't feel quite right effectively re-using those first 32 bytes it _should_ be safe, it's not like this is an XOR.

It also frees up some "spare" bytes at key-change time, so can probably look at building in some backtracking protections

8 months agoAdd comments
B Tasker []
Add comments

8 months agoThose are no longer required
B Tasker []
Those are no longer required

8 months agoWe don't really care why a seed read fails - should just note it has and continue on our way
B Tasker []
We don't really care why a seed read fails - should just note it has and continue on our way

8 months agoUse a couple of threads for RNG generation to improve read speed out of the pipe. Otherwise we quickly exhaust the queue
B Tasker []
Use a couple of threads for RNG generation to improve read speed out of the pipe. Otherwise we quickly exhaust the queue

pi@nextcloudpi:~ $ dd if=/tmp/csprng of=foo count=20K
^C1246+502 records in
1648+0 records out
843776 bytes (844 kB, 824 KiB) copied, 3.91601 s, 215 kB/s

8 months agoAdd comments
B Tasker []
Add comments

8 months agoSwitch to a local seed source.
B Tasker []
Switch to a local seed source.

Currently this is a FIFO written to by another of my scripts - the aim is to be able to use them together, but on other systems /dev/random should be fine as a source too

8 months agoRestructure so we fetch a new seed at startup rather than using the hardcoded one (now removed)
B Tasker []
Restructure so we fetch a new seed at startup rather than using the hardcoded one (now removed)

8 months agoSet up re-seed thread.
B Tasker []
Set up re-seed thread.

For now I've used a requests.get call - this isn't because I'm insane (thought that might also be true), but purely for convenience for now. It'll be amended to read from a *local* source later

8 months agoThat function name might get confusing in a min
B Tasker []
That function name might get confusing in a min

8 months agoJoin the bytes together when pushing to the buffer - helps with read speed at no real cost to us
B Tasker []
Join the bytes together when pushing to the buffer - helps with read speed at no real cost to us

8 months agoMove over to using threads.
B Tasker []
Move over to using threads.

Still no seeding yet, but output can be read from /tmp/csprng

8 months agoRe-seed part way through.
B Tasker []
Re-seed part way through.

Next step then really is to turn this into a thread so that we're constantly generating random data and re-seeding

8 months agoAdd support for a "prediction resistant" mode.
B Tasker []
Add support for a "prediction resistant" mode.

If two instances were started with the same seed (which they will atm, because it's hardcoded) they could leak numbers - one using numbers publicly, the other using the same numbers for private keys etc

Prediction Resistance makes the output non-deterministic, using RDRAND as a source (where possible) and falling back on `get_random_bytes()` (cos I'm building this on a Pi at the moment).

This is basically a fairly loose interpretation of what Amazon did with [S2N](https://aws.amazon.com/blogs/opensource/better-random-number-generation-for-openssl-libc-and-linux-mainline/)

8 months agoAh, my bad - the class doesn't complain if it's not, but the nonce needs to be bytes and I was passing a string
B Tasker []
Ah, my bad - the class doesn't complain if it's not, but the nonce needs to be bytes and I was passing a string

Looks happier

pi@nextcloudpi:~ $ cat op | ent
Entropy = 7.933315 bits per byte.

Optimum compression would reduce the size
of this 144000 byte file by 0 percent.

Chi square distribution for 144000 samples is 12993.14, and randomly
would exceed this value less than 0.01 percent of the times.

Arithmetic mean value of data bytes is 127.1472 (127.5 = random).
Monte Carlo value for Pi is 3.192000000 (error 1.60 percent).
Serial correlation coefficient is 0.005622 (totally uncorrelated = 0.0).

8 months agoSet the nonce by using a counter.
B Tasker []
Set the nonce by using a counter.

ChaCha20 requires a 96bit nonce. Re-use of nonces is fine _if_ the key is different

Although... `ent` doesn't seem to think much of this change

pi@nextcloudpi:~ $ cat op | ent
Entropy = 5.860996 bits per byte.

Optimum compression would reduce the size
of this 144000 byte file by 26 percent.

Chi square distribution for 144000 samples is 534949.35, and randomly
would exceed this value less than 0.01 percent of the times.

Arithmetic mean value of data bytes is 132.1786 (127.5 = random).
Monte Carlo value for Pi is 3.125166667 (error 0.52 percent).
Serial correlation coefficient is -0.028473 (totally uncorrelated = 0.0).

8 months agoAdd some comments
B Tasker []
Add some comments

8 months agoNot using those any more, remove them
B Tasker []
Not using those any more, remove them

8 months agoAdd the CSPRNG I've been building so far in my notes file
B Tasker []
Add the CSPRNG I've been building so far in my notes file

pi@nextcloudpi:~ $ cat op | ent
Entropy = 7.998680 bits per byte.

Optimum compression would reduce the size
of this 144000 byte file by 0 percent.

Chi square distribution for 144000 samples is 263.56, and randomly
would exceed this value 34.29 percent of the times.

Arithmetic mean value of data bytes is 127.5334 (127.5 = random).
Monte Carlo value for Pi is 3.162000000 (error 0.65 percent).
Serial correlation coefficient is -0.002983 (totally uncorrelated = 0.0).

8 months agoSet up repo
B Tasker []
Set up repo