Commit e2119f83 authored by Jeremy Soller's avatar Jeremy Soller
Browse files

Merge branch 'mggmuggins-scheme-example' into 'master'

Rewrite compiling a program and scheme example

See merge request !174
parents 2f264ce9 43b708e9
Pipeline #9293 passed with stage
in 4 minutes and 36 seconds
......@@ -47,8 +47,7 @@
- [Extrautils](./ch04-23-extrautils.md)
- [Development in user space](./ch05-01-user-space.md)
- [Writing an application for Redox](./ch05-02-writing-application.md)
- [Compiling your program](./ch05-03-compiling-program.md)
- [Including a Program in a Redox Build](./ch05-03-compiling-program.md)
- [Ion](./ch05-04-ion.md)
- [What Ion is](./ch05-05-what-ion-is.md)
- [The Manual](./ch05-06-the-manual.md)
......
An example.
===========
# An example.
Enough theory! Time for an example.
We will implement a scheme which holds a vector, and push elements when you write, and pop them when you read. Let's call it `vec:`.
We will implement a scheme which holds a vector. The scheme will push elements
to the vector when it receives writes, and pop them when it is read. Let's call
it `vec:`.
Let's get going:
The complete source for this example can be found at
[redox-os/vec_scheme_example](https://gitlab.redox-os.org/redox-os/vec_scheme_example).
The code
--------
## Setup
So first of all we need to import the things, we need:
In order to build and run this example in a Redox environment, you'll need to
be set up to compile the OS from source. The process for getting a program
included in a local Redox build is laid out in
[chapter 5](ch05-03-compiling-program.md). Pause here and follow that guide if
you want to get this example running.
```rust
extern crate syscall; // add "redox_syscall: "*" to your cargo dependencies
This example assumes that `vec` was used as the name of the crate instead of
`helloworld`. The crate should therefore be located at
`cookbook/recipes/vec/source`.
use syscall::scheme::SchemeMut;
use syscall::error::{Error, Result, ENOENT, EBADF, EINVAL};
Modify the `Cargo.toml` for the `vec` crate so that it looks something like
this:
```toml
[package]
name = "vec"
version = "0.1.0"
edition = "2018"
[[bin]]
name = "vec_scheme"
path = "src/scheme.rs"
[[bin]]
name = "vec"
path = "src/client.rs"
[dependencies]
redox_syscall = "^0.2.6"
```
Notice that there are two binaries here. We'll need another program to interact with
our scheme, since CLI tools like `cat` use more operations than we strictly
need to implement for our scheme. The client uses only the standard library.
## The Scheme Daemon
Create `src/scheme.rs` in the crate. Start by `use`ing a couple of symbols.
```rust
use std::cmp::min;
use std::fs::File;
use std::io::{Read, Write};
use syscall::Packet;
use syscall::scheme::SchemeMut;
use syscall::error::Result;
```
We start by defining our mutable scheme struct, which will implement the `SchemeMut` trait and hold the state of the scheme.
We start by defining our mutable scheme struct, which will implement the
`SchemeMut` trait and hold the state of the scheme.
```rust
struct VecScheme {
......@@ -37,93 +76,189 @@ impl VecScheme {
}
```
First of all we implement, `open()`. Let it accept a reference, which will be the initial content of the vector.
Before implementing the scheme operations on our scheme struct, let's breifly
discuss the way that this struct will be used. Our program (`vec_scheme`) will
create the `vec` scheme by opening the corresponding scheme handler in the root
scheme (`:vec`). Let's implement a `main()` that intializes our scheme struct
and registers the new scheme:
```rust
fn main() {
let mut scheme = VecScheme::new();
let mut handler = File::create(":vec")
.expect("Failed to create the vec scheme");
}
```
When other programs open/read/write/etc against our scheme, the
Redox kernel will make those requests available to our program via this
scheme handler. Our scheme will read that data, handle the requests, and send
responses back to the kernel by writing to the scheme handler. The kernel will
then pass the results of operations back to the caller.
```rust
fn main() {
// ...
let mut packet = Packet::default();
loop {
// Wait for the kernel to send us requests
let read_bytes = handler.read(&mut packet)
.expect("vec: failed to read event from vec scheme handler");
if read_bytes == 0 {
// Exit cleanly
break;
}
// Scheme::handle passes off the info from the packet to the individual
// scheme methods and writes back to it any information returned by
// those methods.
scheme.handle(&mut packet);
handler.write(&packet)
.expect("vec: failed to write response to vec scheme handler");
}
}
```
Now let's deal with the specific operations on our scheme. The
`scheme.handle(...)` call dispatches requests to these methods, so that we
don't need to worry about the gory details of the `Packet` struct.
In most Unix systems (Redox included!), a program needs to open a file before it
can do very much with it. Since our scheme is just a "virtual filesystem",
programs call `open` with the path to the "file" they want to interact with
when they want to start a conversation with our scheme.
Note that we do ignore `flags`, `uid` and `gid`.
For our vec scheme, let's push whatever path we're given to the vec:
```rust
impl SchemeMut for VecScheme {
fn open(&mut self, path: &[u8], _flags: usize, _uid: u32, _gid: u32) -> Result<usize> {
self.vec.extend_from_slice(path);
Ok(path.len())
fn open(&mut self, path: &str, _flags: usize, _uid: u32, _gid: u32) -> Result<usize> {
self.vec.extend_from_slice(path.as_bytes());
Ok(0)
}
}
```
So, now we implement read:
Say a program calls `open("vec:/hello")`. That call will work it's way through
the kernel and end up being dispatched to this function through our
`Scheme::handle` call.
The usize that we return here will be passed back to us as the `id` parameter of
the other scheme operations. This way we can keep track of different open files.
In this case, we won't make a distinction between two different programs talking
to us and simply return zero.
Similarly, when a process opens a file, the kernel returns a number (the file
descriptor) that the process can use to read and write to that file. Now let's
implement the read and write operations for `VecScheme`.
```rust
impl SchemeMut for VecScheme {
// ...
// Fill up buf with the contents of self.vec, starting from self.buf[0].
// Note that this reverses the contents of the Vec.
fn read(&mut self, _id: usize, buf: &mut [u8]) -> Result<usize> {
let res = min(buf.len(), self.vec.len());
let num_written = min(buf.len(), self.vec.len());
for b in buf {
*b = if let Some(x) = self.vec.pop() {
x
if let Some(x) = self.vec.pop() {
*b = x;
} else {
break;
}
}
Result::Ok(res)
Ok(num_written)
}
// Simply push any bytes we are given to self.vec
fn write(&mut self, _id: usize, buf: &[u8]) -> Result<usize> {
for i in buf {
self.vec.push(*i);
}
Ok(buf.len())
}
}
```
Now, we will add `write`, which will simply push to the vector:
Note that each of the methods of the `SchemeMut` trait provide a default
implementation. These will all return errors since they are essentially
unimplemented. There's one more method we need to implement in order to prevent
errors for users of our scheme:
```rust
fn write(&mut self, _id: usize, buf: &[u8]) -> Result<usize> {
for &i in buf {
self.vec.push(i);
}
impl SchemeMut for VecScheme {
// ...
Result::Ok(buf.len())
fn close(&mut self, _id: usize) -> Result<usize> {
Ok(0)
}
}
```
In both our read and write implementation we ignored the `id` parameter for simplicity sake.
Most languages' standard libraries call close automatically when a file object
is destroyed, and Rust is no exception.
To see all the possitble operations on schemes, check out the
[API docs](https://docs.rs/redox_syscall/).
Note that leaving out the missing methods results in errors, when calling them, because of its default implementation.
## A Simple Client
Last, we need the `main` function:
As mentioned earlier, we need to create a very simple client in order to use our
scheme, since some command line tools (like `cat`) use operations other than
open, read, write, and close. Put this code into `src/client.rs`:
```rust
use std::fs::File;
use std::io::{Read, Write};
fn main() {
use syscall::data::Packet;
use std::fs::File;
use std::io::{Read, Write};
use std::mem::size_of;
let mut vec_file = File::open("vec:/hi")
.expect("Failed to open vec file");
let mut scheme = VecScheme::new();
// Create the handler
let mut socket = File::create(":vec").unwrap();
loop {
let mut packet = Packet::default();
while socket.read(&mut packet).unwrap() == size_of::<Packet>() {
scheme.handle(&mut packet);
socket.write(&packet).unwrap();
}
}
vec_file.write(b" Hello")
.expect("Failed to write to vec:");
let mut read_into = String::new();
vec_file.read_to_string(&mut read_into)
.expect("Failed to read from vec:");
println!("{}", read_into); // olleH ih/
}
```
How to compile and run this example
-----------------------------------
For the time being there is no easy way to compile and deploy binaries for/on the redox platform.
We simply open some "file" in our scheme, write some bytes to it, read some
bytes from it, and then spit those bytes out on stdout. Remember, it doesn't
matter what path we use, since all our scheme does is add that path to the vec.
In this sense, the vec scheme implements a global vector.
There is however a way and it goes as follows:
## Running the Scheme
1. Add your folder (as a soft link) into your local Redox repository (e.g. schemes/);
2. Add your binary to the root Makefile (e.g. under the schemes target);
3. Compile Redox via `make`;
Since we've already set up the program to build and run in the redox VM,
simply run:
- `touch filesystem.toml`
- `make qemu`
Note: Do **not** add your folder to the git history, this 3-step process is only meant for local testing purposes. Also make sure to name your folder and binary as specified in your binary's `cargo.toml`;
We'll need multiple terminal windows open in the QEMU window for this step.
Notice that both binaries we defined in our `Cargo.toml` can now be found in
`file:/bin` (`vec_scheme` and `vec`). In one terminal window, run
`sudo vec_scheme`. A program needs to run as root in order to register a new
scheme. In another terminal, run `vec` and observe the output.
Now, go ahead and test it by running `make qemu` (or a variant)!
## Exercises for the reader
Exercises for the reader
------------------------
- Make the vec scheme print out something whenever it gets events. For example,
print out the user and group ids of the user who tries to open a file in the
scheme.
- Create a unique vec for each opened file in your scheme. You might find a
hashmap useful for this.
- Write a scheme that can run code for your favorite esoteric programming
language.
+ Compile and run the `VecScheme` example on the Redox platform;
+ There is also a Scheme trait, which is immutable. Come up with some use cases for this one;
+ Write a scheme that can run code for your favorite esoteric programming language;
# Writing an application for Redox
# Compiling your program
# Including a Program in a Redox Build
Thanks to redox' cookbook, building programs is a snap. In this example, we will be building the helloworld program that `cargo new` automatically generates.
Redox's cookbook makes packaging a program to include in a build fairly
straightforward. This example walks through adding the "hello world"
program that `cargo new` automatically generates to a local build of the
operating system.
## Step One: Setting up your program
This process is largely the same for other rust crates and even non-rust
programs.
To begin, go to your projects directory, here assumed to be `~/Projects/`. Open the terminal, and run `cargo new helloworld --bin`. For reasons that will become clear later,
you must make your program compile from a git repository, so run `cd helloworld;git init` to make helloworld a git repo, and `git status` to see which files you need to add to the repo. You should see something like this
## Step One: Setting up the recipe
```bash
On branch master
The cookbook will only build programs that have a recipe defined in
`cookbook/recipes`. To create a recipe for hello world, first create a
directory `cookbook/recipes/helloworld`. Inside this directory create a file
`recipe.toml` and add these lines to it:
Initial commit
```toml
[build]
template = "cargo"
```
Untracked files:
(use "git add <file>..." to include in what will be committed)
The `[build]` section defines how cookbook should build our project. There are
several templates but `"cargo"` should be used for rust projects.
.gitignore
Cargo.toml
src/
Cookbook will fetch the sources for a program from a git or tarball URL
specified in the `[source]` section of this file if
`cookbook/recipes/program_name/source` does not exist, and will also fetch
updates when running `make fetch`.
nothing added to commit but untracked files present (use "git add" to track)
```
For this example, there is no upstream URL to fetch the sources from, hence no
`[source]` section. Instead, we will simply develop in the `source` directory.
Add all the files by running `git add .gitignore Cargo.toml src/` and commit by running `git commit -m "Added files to helloworld."`.
## Step Two: Writing the program
There is only one more thing that must be done to set up your program. Go to the `Cargo.toml` of your project and add
Since this is a hello world example, this step is very straightforward. Simply
create `cookbook/recipes/helloworld/source`. In that directory, run `cargo
init --name="helloworld"`.
```toml
[[bin]]
name = "helloworld"
path = "src/main.rs"
```
to the bottom. Now the program is sufficiently set up, and it is ready to be added to your redox build.
For cargo projects that already exist, either include a URL to the git
repository in the recipe and let cookbook pull the sources automatically during
the first build, or simply copy the sources into the `source` directory.
## Step Two: Adding your program to your redox build
## Step Three: Add the program to the redox build
To be able to access your program from redox, it must be added to both the build process and the filesystem. Adding your program to the filesystem is easy: go to your `redox/filesystem.toml` file, and look under the `[packages]` table. It should look like this:
To be able to access a program from within Redox, it must be added to the
filesystem. Open `redox/filesystem.toml` and find the `[packages]` table.
During the filesystem (re)build, the build system uses cookbook to package all
the applicationsin this table, and then installs those packages to the new
filesystem. Simply add `helloworld = {}` anywhere in this table.
```toml
[packages]
......@@ -45,21 +57,9 @@ contain = {}
coreutils = {}
#dash = {}
extrautils = {}
#games = {}
#gcc = {}
#gnu-binutils = {}
#gnu-make = {}
installer = {}
ion = {}
#lua = {}
netstack = {}
netutils = {}
#newlib = {}
orbdata = {}
orbital = {}
orbterm = {}
orbutils = {}
#pixelcannon = {}
#
# 100+ omitted for brevity
#
pkgutils = {}
ptyd = {}
randd = {}
......@@ -68,33 +68,24 @@ redoxfs = {}
smith = {}
#sodium = {}
userutils = {}
```
Under `userutils = {}` add a line for your own program:
```toml
userutils = {}
# Add this line:
helloworld = {}
```
Now when building the filesystem, redox will look for a `helloworld` binary.
With the file system in order, you can now add your program to the build process by adding a recipie. Spoilers: this is the easy part.
Under `redox/cookbook/recipes/`, make a new directory called helloworld. In helloworld, create a file called `recipe.sh`.
Remember the git repository `~/Projects/helloworld/`? Its about to be relevant. In the file `recipe.sh`, write
```bash
GIT=~/Projects/helloworld/
```
With that, helloworld will now be built with and accessible from redox.
In order to rebuild the filesystem image to reflect changes in the `source`
directory, it is nessesary to run `touch filesystem.toml` before running make.
## Step Three: Running your program
Go up to your `redox/` directory and run `make all`. Once it finishes running, run `make qemu`, log in to redox, open the terminal, and run `helloworld.` It should print
Go up to your `redox/` directory and run `make all`. Once the rebuild is
finished, run `make qemu`, log in to Redox, open the terminal, and run
`helloworld`. It should print
```shell
Hello, world!
```
Note that the `helloworld` binary can be found in `file:/bin` in the VM (`ls
file:/bin`).
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment