Fix sbrk to actually return the *old* break.

 Use new allocator API, and otherwise update for new Rust

- New allocator API
- Remove the "allocator" feature, which should be unnecessary due to how
the new allocator API works
- NonZero no longer implements Deref (https://github.com/rust-lang/rust/pull/41064)
- NonZero::new() returns an Option; use NonZero::new_unchecked()
- Thread locals are no longer 'static (https://github.com/rust-lang/rust/pull/43746)
- Changes to feature flags
- Use unsafe to access extern static (https://github.com/rust-lang/rust/issues/36247)

MacOSX is not supported anymore

Grammar fix

it is missing some needed syscalls.

fixes #37

by switching from the syscall crate to the sc crate. The sc crate is a
fork of the syscall crate that adds supports for several other
platforms.

* fix the build

the `unborrow!` macro is need in the block and bookkeeper modules

* cfg(tls) away impl LocalAllocator

as the struct LocalAllocator is not available when the tls feature is
disabled

To avoid linking hell.

David Teller authored 8 years ago and Jeremy Soller committed 8 years ago

This feature is now in Rust stable, so adding !#[feature(question_mark)] is now useless and causes a warning.

We make the `Leak` trait unsafe and consituting an invariant. Secondly,
we implement `Leak` for all `Copy` types.

We equip all `unsafe` blocks with a timestamped confirmations of an
audit. Whenever a block is changed, it should be removed until a new
audit is made.

The format of these messages is:

```
LAST AUDIT: yyyy-mm-dd (Username)
```

Close #28.

Various issues (like premature checks) were present in the byte count
beforehand. We fix these.

- Fix invalid byte counts.
- Add a `LOCAL_MEMTRIM_STOP` constant to `shim`, which serves for the
  minimum memtrimmed allocator (i.e. it will stop when the byte count is
  below this value).
- Add more logs.
- Add assertions about libc interference with SBRK.

There is still some problem when running

This commit is a complete revision of `shim`. Anything which is
configurable of nature is now moved to `shim` allowing complete
customizability of `ralloc`.

But it doesn't end there!

The logs got quite messy before due to the classical TMI problem. We
solve this by adding multiple "log levels" each describing a different
class of importance.

- Add multi-level logging.
- Move each component of shim into distinct modules.
- Add allocator canonicalization to `shim`.
- Add adjustable constant to `shim`.
- Add default OOM handler to `shim`.
- Add logging to everything.
- Remove the `libc_write` module and add it to `log` instead.
- Add colourful OOM message.
- Tweak the allocator canonicalization strategy.
- Rename the `log!` macro to `bk_log!`, and use the old name for the
  generalized logging macro.
- Remove the `unsafe_no_brk_lock` feature, due to being unsafe in
  multithreaded environment (and in singlethreaded ones,
  `unsafe_no_mutex_lock` is sufficient).
- Remove the `sys` module in favor of direct `shim` imports.

This commit introduces OS/BRK memtrimming, in which the memory from the
global allocator is freed to the OS when a certain limit is reached.
This avoids long-running processes from acquiring too much memory from
the system and never releasing it again.

- Implement safe BRK releasing.

Previously, BRK was making two syscalls in glibc due to requesting
the program break and then setting it. This is fixed by caching this
program break.

This commit prepares for OS memtrimming, which needs a well-defined
thread-safe function.

- Completely remove libc dependency due to allocation behavior being
  uncertain in non-glibc implementation (the POSIX standard tells little
  to nothing about this).
- Move the default OOM handler to shim.
- Give an error message in the default OOM handler.
- Add SBRK benchmark.

This should improve performance of fresh allocations by 30-40%.

Memtrimming (releasing memory to a higher allocator or the OS) is
important for avoiding accumulation. This commit introduces memtrimming
for local allocators to avoid fragmentation and stubs accumulating in
local allocators.

By keeping track of how many bytes that are currently in the bookkeeper,
we can estimate the level of fragmentation and thus decide if we should
memtrim (i.e. free to the global allocator).

The next thing is to implement global memtrimming.

We do a few minor changes as well:

- Refactor for_each to use a new method, `pop_iter` a iterator popping
  elements of the vector.
- Introduce a new field `total_bytes` keeping track of the number of
  bytes in the bookkeeper.
- Move the initializers of the allocators to methods defined on the type
  itself.
- Introduce an `on_new_memory()` method, which is called whenever new
  memory is introduced.

Previously, `push` might oversee a possible merge and thus increasing fragmentation. This commit (by @nilset) fixes that.

- Ignore slow tests.

Patch by @nilset.

Requested on Reddit.

We add first-class debugger support by providing functions for marking memory blocks uninitialized/free to the debugger. The default implementation is wired to valgrind, but it can easily be replaced.

This feature was requested by @jackpot51, and will be used until Redox gets full TLS support.

Apparently, atomically reference counted objects would allow destructors to run after the allocator had been deinitialized. To fix this, we make sure the allocator is never disabled, but simply will fallback to the global alloctor when deinitialized. This also fixes an important bug in TLS keys (i.e. the order of the thread dtors were unknown).

We introduce a few other smaller changes as well:

- Use uniform convention for todos. Add a colon after the TODO, and start with capital letter, and end with punctuation.

- Fix broken tests which would free a larger buffer than allocated.

- Remove debug_tools in favor of planned Valgrind compatibility.

- Remove `LazyInit`'s unreachable state.

- Replace `get_allocator` with a macro (due to the lack of generic closures).

- Let `Writer` hold the lock.

- Rename `Writer` to `LogWriter`

- Fix warnings in tests.

- Remove forgotten file.

It is generally considered better to do so, due to the more descriptive error messages.

- Remove warnings from the minimal.rs test

Weird stuff. Ain't high, ain't drunk, but shit's happening, and it is fucked up. I can smell rainbows.