Superior Word Designator Support

README.md

@@ -5,7 +5,11 @@
 [![crates.io](http://meritbadge.herokuapp.com/ion-shell)](https://crates.io/crates/ion-shell)
 ![LOC](https://tokei.rs/b1/github/redox-os/ion)
 
-# New Ion MdBook
+> Ion is still a WIP, and both it's syntax and rules are subject to change over time. It is
+> still quite a ways from becoming stabilized, but we are getting close. Changes to the
+> syntax at this time are likely to be minimal.
+
+# Ion Manual
 
 We are providing our manual for Ion in the form of a markdown-based book, which is accessible via:
 
@@ -13,6 +17,16 @@ We are providing our manual for Ion in the form of a markdown-based book, which
 - Installing the mdbook via our `setup.ion` script and having Ion open an offline copy via `ion-docs`.
 - Building and serving the book in the **manual** directory yourself with [mdBook](https://github.com/azerupi/mdBook)
 
+> Note, however, that the manual is incomplete, and does not cover all of Ion's functionality
+> at this time. Anyone willing to help with documentation should request to do so in the chatroom.
+
+# Contributors
+
+Send an email to [info@redox-os.org](mailto:info@redox-os.org) to request invitation for joining
+the developer chatroom for Ion. Experience with Rust is not required for contributing to Ion. There
+are ways to contribute to Ion at all levels of experience, from writing scripts in Ion and reporting
+issues, to seeking mentorship on how to implement solutions for specific issues on the issue board.
+
 # Introduction
 
 Ion is a modern system shell that features a simple, yet powerful, syntax. It is written entirely
@@ -40,27 +54,6 @@ class variables with their own unique **@** sigil. Strings are also treated as f
 variables with their own unique **$** sigil. Both support being sliced with **[range]**, and they
 each have their own supply of methods.
 
-# Why Not POSIX?
-
-If Ion had to follow POSIX specifications, it wouldn't be half the shell that it is today, and
-there'd be no solid reason to use Ion over any other existing shell, given that it'd basically be
-the same as every other POSIX shell. Redox OS itself doesn't follow POSIX specifications, and
-neither does it require a POSIX shell for developing Redox's userspace. It's therefore not meant
-to be used as a drop-in replacement for Dash or Bash. You should retain Dash/Bash on your system
-for execution of Dash/Bash scripts, but you're free to write new scripts for Ion, or use Ion as
-the interactive shell for your user session. Redox OS, for example, also contains Dash for
-compatibility with software that depends on POSIX scripts.
-
-That said, Ion's foundations are heavily inspired by POSIX shell syntax. If you have experience
-with POSIX shells, then you already have a good idea of how most of Ion's core features operate. A
-quick sprint through this documentation will bring you up to speed on the differences between our
-shell and POSIX shells. Namely, we carry a lot of the same operators: **$**, **|**, **||**, **&**,
-**&&**, **>**, **<**, **<<**, **<<<**, **$()**, **$(())**.  Yet we also offer some functionality
-of our own, such as **@**, **@()**, **$method()**, **@method()**, **^|**, **^>**, **&>**, **&|**.
-Essentially, we have taken the best components of the POSIX shell specifications, removed the bad
-parts, and implemented even better features on top of the best parts. That's how open source
-software evolves: iterate, deploy, study, repeat.
-
 # Compile / Install Instructions
 
 Rust nightly is required for compiling Ion. Simplest way to obtain Rust/Cargo is by
@@ -69,14 +62,6 @@ not ship Rust natively, or if you want more flexibility in Rust compilation capa
 
 Then, it's just a matter of performing one of the following methods:
 
-## Install Latest Stable Version From Crates.io
-
-Use the `--force` flag when updating a binary that's already installed with cargo.
-
-```sh
-cargo install ion-shell
-```
-
 ## Install Direct From Git
 
 ```sh
@@ -87,8 +72,7 @@ cargo install --git https://github.com/redox-os/ion/
 
 ```sh
 git clone https://github.com/redox-os/ion/
-cd ion
-cargo build --release
+cd ion && cargo build --release
 ```
 
 # Git Plugin

src/shell/binary/designators.rs

+use parser::ArgumentSplitter;
+use shell::Shell;
+use std::borrow::Cow;
+use std::str;
+
+bitflags! {
+    struct Flags: u8 {
+        const DQUOTE = 1;
+        const SQUOTE = 2;
+        const DESIGN = 4;
+    }
+}
+
+#[derive(Debug)]
+enum Token<'a> {
+    Designator(&'a str),
+    Text(&'a str),
+}
+
+struct DesignatorSearcher<'a> {
+    data:  &'a [u8],
+    flags: Flags,
+}
+
+impl<'a> DesignatorSearcher<'a> {
+    fn new(data: &'a [u8]) -> DesignatorSearcher {
+        DesignatorSearcher {
+            data,
+            flags: Flags::empty(),
+        }
+    }
+
+    fn grab_and_shorten(&mut self, id: usize) -> &'a str {
+        let output = unsafe { str::from_utf8_unchecked(&self.data[..id]) };
+        self.data = &self.data[id..];
+        output
+    }
+}
+
+impl<'a> Iterator for DesignatorSearcher<'a> {
+    type Item = Token<'a>;
+
+    fn next(&mut self) -> Option<Token<'a>> {
+        let mut iter = self.data.iter().enumerate();
+        while let Some((id, byte)) = iter.next() {
+            match *byte {
+                b'\\' => {
+                    let _ = iter.next();
+                }
+                b'"' if !self.flags.contains(Flags::SQUOTE) => self.flags ^= Flags::DQUOTE,
+                b'\'' if !self.flags.contains(Flags::DQUOTE) => self.flags ^= Flags::SQUOTE,
+                b'!' if !self.flags.intersects(Flags::DQUOTE | Flags::DESIGN) => {
+                    self.flags |= Flags::DESIGN;
+                    if id != 0 {
+                        return Some(Token::Text(self.grab_and_shorten(id)));
+                    }
+                }
+                b' ' | b'\t' | b'\'' | b'"' | b'a'...b'z' | b'A'...b'Z'
+                    if self.flags.contains(Flags::DESIGN) =>
+                {
+                    self.flags ^= Flags::DESIGN;
+                    return Some(Token::Designator(self.grab_and_shorten(id)));
+                }
+                _ => (),
+            }
+        }
+
+        if self.data.is_empty() {
+            None
+        } else {
+            let output = unsafe { str::from_utf8_unchecked(&self.data) };
+            self.data = b"";
+            Some(if self.flags.contains(Flags::DESIGN) {
+                Token::Designator(output)
+            } else {
+                Token::Text(output)
+            })
+        }
+    }
+}
+
+pub(crate) fn expand_designators<'a>(shell: &Shell, cmd: &'a str) -> Cow<'a, str> {
+    let context = shell.context.as_ref().unwrap();
+    if let Some(buffer) = context.history.buffers.iter().last() {
+        let buffer = buffer.as_bytes();
+        let buffer = unsafe { str::from_utf8_unchecked(&buffer) };
+        let mut output = String::with_capacity(cmd.len());
+        for token in DesignatorSearcher::new(cmd.as_bytes()) {
+            match token {
+                Token::Text(text) => output.push_str(text),
+                Token::Designator(text) => match text {
+                    "!!" => output.push_str(buffer),
+                    "!$" => output.push_str(last_arg(buffer)),
+                    "!0" => output.push_str(command(buffer)),
+                    "!^" => output.push_str(first_arg(buffer)),
+                    "!*" => output.push_str(&args(buffer)),
+                    _ => output.push_str(text),
+                },
+            }
+        }
+        return Cow::Owned(output);
+    }
+
+    Cow::Borrowed(cmd)
+}
+
+fn command<'a>(text: &'a str) -> &'a str { ArgumentSplitter::new(text).next().unwrap_or(text) }
+
+// TODO: do this without allocating a string.
+fn args(text: &str) -> String {
+    ArgumentSplitter::new(text).skip(1).collect::<Vec<&str>>().join(" ")
+}
+
+fn first_arg<'a>(text: &'a str) -> &'a str { ArgumentSplitter::new(text).nth(1).unwrap_or(text) }
+
+fn last_arg<'a>(text: &'a str) -> &'a str { ArgumentSplitter::new(text).last().unwrap_or(text) }

src/shell/binary/mod.rs

 //! Contains the binary logic of Ion.
+mod designators;
 mod prompt;
 mod readln;
 mod terminate;
@@ -122,8 +123,8 @@ impl Binary for Shell {
             if let Some(command) = self.readln() {
                 if !command.is_empty() {
                     if let Ok(command) = self.terminate_quotes(command.replace("\\\n", "")) {
-                        let cmd = command.trim();
-                        self.on_command(cmd);
+                        let cmd: &str = &designators::expand_designators(&self, command.trim());
+                        self.on_command(&cmd);
 
                         if cmd.starts_with('~') {
                             if !cmd.ends_with('/')

src/shell/job.rs

-use std::fs::File;
-use std::process::{Command, Stdio};
-
-// use glob::glob;
-
 use super::Shell;
-use parser::ArgumentSplitter;
 use parser::expand_string;
 use parser::pipelines::RedirectFrom;
 use smallstring::SmallString;
+use std::fs::File;
+use std::process::{Command, Stdio};
 use std::str;
 use types::*;
 
@@ -43,76 +39,12 @@ impl Job {
         let mut expanded = Array::new();
         expanded.grow(self.args.len());
         expanded.extend(
-            self.args
-                .drain()
-                .flat_map(|arg| match arg.as_str() {
-                    "!!" => expand_last_command(shell, Operation::All),
-                    "!$" => expand_last_command(shell, Operation::LastArg),
-                    "!0" => expand_last_command(shell, Operation::Command),
-                    "!^" => expand_last_command(shell, Operation::FirstArg),
-                    "!*" => expand_last_command(shell, Operation::NoCommand),
-                    _ => expand_arg(&arg, shell),
-                })
-                .filter(|x| !x.is_empty()),
+            self.args.drain().flat_map(|arg| expand_arg(&arg, shell)).filter(|x| !x.is_empty()),
         );
         self.args = expanded;
     }
 }
 
-pub(crate) enum Operation {
-    LastArg,
-    FirstArg,
-    Command,
-    NoCommand,
-    All,
-}
-
-/// Expands the last command that was provided to the shell.
-///
-/// If `last_arg` is set to `true`, then only the last argument of
-/// the last command will be expanded.
-pub(crate) fn expand_last_command(shell: &Shell, operation: Operation) -> Array {
-    fn get_last_arg(buffer: &str) -> &str { ArgumentSplitter::new(buffer).last().unwrap_or(buffer) }
-
-    fn get_first_arg(buffer: &str) -> &str {
-        ArgumentSplitter::new(buffer).skip(1).next().unwrap_or(buffer)
-    }
-
-    fn get_command(buffer: &str) -> &str { ArgumentSplitter::new(buffer).next().unwrap_or(buffer) }
-
-    fn get_args(buffer: &str) -> &str {
-        let bbuffer = buffer.as_bytes();
-        if let Some(pos) = bbuffer.iter().position(|&x| x == b' ') {
-            let buffer = &bbuffer[pos + 1..];
-            if let Some(pos) = buffer.iter().position(|&x| x != b' ') {
-                return unsafe { str::from_utf8_unchecked(&buffer[pos..]) };
-            }
-        }
-
-        buffer
-    }
-
-    fn expand_args(buffer: &str, shell: &Shell) -> Array {
-        ArgumentSplitter::new(buffer).flat_map(|b| expand_arg(b, shell)).collect::<Array>()
-    }
-
-    if let Some(ref context) = shell.context {
-        if let Some(buffer) = context.history.buffers.iter().last() {
-            let buffer = buffer.as_bytes();
-            let buffer = unsafe { str::from_utf8_unchecked(&buffer) };
-            return match operation {
-                Operation::LastArg => expand_arg(get_last_arg(buffer), shell),
-                Operation::FirstArg => expand_arg(get_first_arg(buffer), shell),
-                Operation::Command => expand_arg(get_command(buffer), shell),
-                Operation::NoCommand => expand_args(get_args(buffer), shell),
-                Operation::All => expand_args(buffer, shell),
-            };
-        }
-    }
-
-    array![""]
-}
-
 /// Expands a given argument and returns it as an `Array`.
 fn expand_arg(arg: &str, shell: &Shell) -> Array {
     let res = expand_string(&arg, shell, false);