ion issueshttps://gitlab.redox-os.org/redox-os/ion/-/issues2023-11-15T12:30:19Zhttps://gitlab.redox-os.org/redox-os/ion/-/issues/1032unexplained attempt to open file2023-11-15T12:30:19ZRon Williamsunexplained attempt to open fileRunning `ls foo`, where foo does not exist, seems to result in an attempt to open a file named `ls foo`. I'm not sure if this is an attempt to execute or just an attempt to test the existence of the file `ls foo`. Either way, it doesn't ...Running `ls foo`, where foo does not exist, seems to result in an attempt to open a file named `ls foo`. I'm not sure if this is an attempt to execute or just an attempt to test the existence of the file `ls foo`. Either way, it doesn't seem like the right thing to do.https://gitlab.redox-os.org/redox-os/ion/-/issues/1030Proposal: Convert MakeFiles and bash scripts into rust via "cargo xtask" conv...2023-03-28T09:02:39ZFlorian NaumannProposal: Convert MakeFiles and bash scripts into rust via "cargo xtask" convention.# Problem with make and bash
Ion shell grows more and more in complexity and so does the local automation system.
Make and bash becomes harder and harder to extend on for automation.
While make and bash might have been sufficient for a ...# Problem with make and bash
Ion shell grows more and more in complexity and so does the local automation system.
Make and bash becomes harder and harder to extend on for automation.
While make and bash might have been sufficient for a starting point to automate, this project has reached the point where those tools are not justifiable anymore in my opinion .
I propose that we change gears and use rust for local automation too by applying the xtask convention.
## What is xtask
There is a simple convention, called xtask, to follow.
This convention is possible via the workspace feature of Rust. The rust code for automation is then just another workspace member. Following this convention, we get a full fledged CLI API for automation which depends only on Rust and Cargo. In this system we could utilize the power of rust and its ecosystem for automation.
[Link](https://github.com/matklad/cargo-xtask) to this convention.
## Arguments for cargo xtask convention over make and bash scripts.
- The Redox-Os ecosystem is also about leveraging the capabilities of Rust, so why not also in the automation system.
- Make is primarily a build system and not a command runner. Cargo already provides an build system for Rust.
- Bash as script language does not scale well with growing complexity in automation. Rust on the other hand is way better in this department.
- Can have one language in the project for the application and for the automation system.
- Not all people come from the Linux realm. Automation via Make and Bash increases the learning curve for those people who want fix/improve automation.
- More flexibility in designing the CLI API compared to Make match rules or Bash "getopts" thing.
- It is advantageous to get rid of make and bash for a latter windows port.
- Reduces build tech stack, taking out make and bash.
## Willingness for the proof of concept
For demo purposes I would be willing provide a PR which implements the unit/integration test system of Ion via rust in xtask convention. If the PR/Proposal convinces the community, then I would be happy to replace all Make and Bash scripts via Rust in the Ion repository.https://gitlab.redox-os.org/redox-os/ion/-/issues/1028Replace and replace_regex should allow empty strings as parameters2023-10-27T22:46:53ZTimofey ProdanovReplace and replace_regex should allow empty strings as parametersBasically, the following method call would work: `$replace("abc" "a" "0")`, while this would fail:
```
echo $replace("abc" "a" "")
ion: expansion error: replace: two arguments are required
```
The same goes for `replace_regex`.
I think ...Basically, the following method call would work: `$replace("abc" "a" "0")`, while this would fail:
```
echo $replace("abc" "a" "")
ion: expansion error: replace: two arguments are required
```
The same goes for `replace_regex`.
I think overall the methods should allow for empty strings as parameters, but check them later and fail, if it is not an appropriate argument.
Compare this to python:
```python
python -c 'print("abc".replace("a", ""))'
```
and to `sed`:
```bash
echo abc | sed 's/a//g'
```
The same issue was raised [here](redox-os/ion#805).
Overall, I can try to fix that later if the change makes sense.https://gitlab.redox-os.org/redox-os/ion/-/issues/1025ion cannot handle wildcards containing scheme prefixes like 'disk:'2023-03-04T22:57:15ZFlorian Naumannion cannot handle wildcards containing scheme prefixes like 'disk:'Ion can not handle paths with prefix for redox schemes.
Ion uses the crate glob for expanding given paths in commands.
While PR https://gitlab.redox-os.org/redox-os/ion/-/merge_requests/1239 as workaround makes ion work with the prefix ...Ion can not handle paths with prefix for redox schemes.
Ion uses the crate glob for expanding given paths in commands.
While PR https://gitlab.redox-os.org/redox-os/ion/-/merge_requests/1239 as workaround makes ion work with the prefix **file:**, it still does not work in general with other prefixes.
Example: file:/path/to/everything/* works but not disk:*.
Problem: glob as crate does not understand the concept of redox os prefixes.
Possible approaches:
- Make an issue or better PR to the crate glob to support redox os prefixes.
- Make a fork of glob and implement it there.
- Make replacement of glob to support prefixes.https://gitlab.redox-os.org/redox-os/ion/-/issues/1024null-coalescing operator2023-03-02T16:22:03ZFlorian Naumannnull-coalescing operatorThere an issue which sparked the idea https://gitlab.redox-os.org/redox-os/ion/-/issues/1001#note_23188
Code example for how an user might use it.
```sh
let array = array ?: [default]
```
This should also work with strings.There an issue which sparked the idea https://gitlab.redox-os.org/redox-os/ion/-/issues/1001#note_23188
Code example for how an user might use it.
```sh
let array = array ?: [default]
```
This should also work with strings.https://gitlab.redox-os.org/redox-os/ion/-/issues/1023ternary operator for arrays/strings2023-03-02T16:16:04ZFlorian Naumannternary operator for arrays/stringsThis issue https://gitlab.redox-os.org/redox-os/ion/-/issues/1001#note_23188 mentioned the idea of an ternary operator for arrays.
This code sample shows how it could be used by the user.
```sh
let array = []
let array = array ? array...This issue https://gitlab.redox-os.org/redox-os/ion/-/issues/1001#note_23188 mentioned the idea of an ternary operator for arrays.
This code sample shows how it could be used by the user.
```sh
let array = []
let array = array ? array : [default]
# alternately
let array = if array; array; else [default]; end
```
This feature should also work with strings in same way.https://gitlab.redox-os.org/redox-os/ion/-/issues/1022Enrich error messages information where the error occurred.2023-03-31T07:37:52ZFlorian NaumannEnrich error messages information where the error occurred.# What I want
Error messages do not show anything where the error has occurred.
Error messages should include information where the error has happened.
- Path to file in which the error occurred.
- The number of the line where the err...# What I want
Error messages do not show anything where the error has occurred.
Error messages should include information where the error has happened.
- Path to file in which the error occurred.
- The number of the line where the error was raised.
- The column at the char position in the line where the error starts.
I think printing out the absolute path of the file is most reliable and help with locating the error the best.
If the command was executed with option "-c" then the path should be a keyword like "literal command".
Because paths can become long I propose that the ion errors should have 2 lines.
First Line shows the error kind and the file path.
Second line shows a tuple (line_number:column) and then the error description.
## Purpose
It would aid when debugging scripts. Especially in more complex scripts this can make life easier.
## Example with a file which is named "some_file"
```sh
echo "Print out not declared variable 'ab' "
echo $ab
```
Print out not declared variable 'ab'
**Desired output:**
```
Print out not declared variable 'ab'
ion: expansion error at (path_to_file):
ion: (2:5) Variable does not exist
```
**Instead of current output:**
```text
Print out not declared variable 'ab'
ion: expansion error: Variable does not exist
```
## Example with inline command via option "-c"
```sh
ion -c "echo $does_not_exits"
```
**Desired output:**
```text
ion: expansion error in literal command:
ion: (1:6)
```https://gitlab.redox-os.org/redox-os/ion/-/issues/1015source-sh complaints and import truncated value on var with new line2022-09-29T04:27:16ZHernawan Faiz Abdillahsource-sh complaints and import truncated value on var with new line# Summary
When executing bash script that produce env var which contains new line char, Ion will complaint about invalid env vars.
Additionally, it also truncate the value because the new line is considered as another env var entry.
# E...# Summary
When executing bash script that produce env var which contains new line char, Ion will complaint about invalid env vars.
Additionally, it also truncate the value because the new line is considered as another env var entry.
# Expected
It should not complaint and not truncate.
# How to reproduce
Create this simple shell export script.
```
#!/bin/sh
export VAR_WITH_NEWLINE=this is an example value which contains some\nnew\nline
```
Then start ion and do `source-sh`
```
# source-sh a.txt
Invalid environment variable 'line'. Proceeding anyway
Invalid environment variable 'new'. Proceeding anyway
Set VAR_WITH_NEWLINE to this is an example value which contains some
```https://gitlab.redox-os.org/redox-os/ion/-/issues/1013quiet option for source-sh2022-09-11T23:34:51ZSteven Peasequiet option for source-shRight now source-sh seems to echo messages like "Set PATH to". It would be nice to able to make this quiet so it doesn't get shown up whenever I open a new terminal when using it with my startup scripts.
I do appreciate the usefulness o...Right now source-sh seems to echo messages like "Set PATH to". It would be nice to able to make this quiet so it doesn't get shown up whenever I open a new terminal when using it with my startup scripts.
I do appreciate the usefulness of source-sh though in making things more backwards compatible with existing shells and third-party shell scripts needed for packages to function (in this case, nix).https://gitlab.redox-os.org/redox-os/ion/-/issues/1012filename unescaping with non-ascii characters2021-09-20T10:53:50ZEnzo Cioppettinifilename unescaping with non-ascii charactersbug: can't open files with non-ascii characters and whitespace in their names
related: none
code: input
```
touch 'fïle 1.txt'
cat fïle\ 1.txt
```
expect: the file contents
result: `cat: 'fïle\ 1.txt': No such file or directory`
ver...bug: can't open files with non-ascii characters and whitespace in their names
related: none
code: input
```
touch 'fïle 1.txt'
cat fïle\ 1.txt
```
expect: the file contents
result: `cat: 'fïle\ 1.txt': No such file or directory`
version: df993477678248a62745c9c0a56c880dfd06b515
context: using ion as a interactive shell, the escaping comes from tab-completion, using single quotes instead of `\` works just fine.
behavior of bash/dash/zsh/fish/oil: open the filehttps://gitlab.redox-os.org/redox-os/ion/-/issues/1011process expansions with fzf stall forever2021-09-20T10:55:43Zunmellowprocess expansions with fzf stall foreverbug: placing fzf anywhere inside process expansions results in stalling
you can ctrl+c to stop it resulting in the expansion becoming empty string
expect: process expansion launches fzf allowing me to use it to select what is out...bug: placing fzf anywhere inside process expansions results in stalling
you can ctrl+c to stop it resulting in the expansion becoming empty string
expect: process expansion launches fzf allowing me to use it to select what is outputted into
a function/executable/variable
related: none
code: input
```
myvar=$(echo "foo
bar" | fzf)
```
expect: output
```
depending on you selection an output of foo/bar/empty string
```
kernel: linux 5.10.47-1-lts
version: ion 1.0.0-alpha (x86_64-unknown-linux-gnu)
interaction: fzf
context: none
behavior of bash
same as expected outputhttps://gitlab.redox-os.org/redox-os/ion/-/issues/1010Is it possible to return arrays from functions?2022-09-11T23:29:48Zb-fuzeIs it possible to return arrays from functions?I'm new to Ion and I was wondering if functions could return complex structures such as arrays and maps instead of just strings?I'm new to Ion and I was wondering if functions could return complex structures such as arrays and maps instead of just strings?https://gitlab.redox-os.org/redox-os/ion/-/issues/1008Alias names cannot containt semicolons2021-09-20T10:56:42ZA1ex-NAlias names cannot containt semicolonsI have aliases in bash and fish that contain semicolons. For example, ":q" for "exit". In ion, this does not work.
`ion: alias name, ':q', is invalid`I have aliases in bash and fish that contain semicolons. For example, ":q" for "exit". In ion, this does not work.
`ion: alias name, ':q', is invalid`https://gitlab.redox-os.org/redox-os/ion/-/issues/1006ArrayMethod and StringMethod implementation should be refactored into a gener...2021-07-14T08:33:11ZNils ErikssonArrayMethod and StringMethod implementation should be refactored into a generic method structurefeat: ArrayMethod and StringMethod are two structures with identical members (StringMethod members are all public).
These two are doing largely the same work (supplying methods to one of the objects, string or array) but the implementat...feat: ArrayMethod and StringMethod are two structures with identical members (StringMethod members are all public).
These two are doing largely the same work (supplying methods to one of the objects, string or array) but the implementation differs completely how the two structures does this. It feels like both of the files were written by two different persons at two different times.
ArrayMethod's methods largely defines each in-shell method in a separate source code method.
StringMethod only has one handle method and uses builtin macros, within the scope of the method. All logic are decided in a huge match block.
The logic behind Array and String methods has to work with the name of the variable directly and resolve it's value manually. Ideally you want to write functions that takes typed arguments which works with values directly and does not have to de reference variable names or initiate array expressions. That should be the job of a parser.
The method structures are kind of parsed, but it could be better.
If we look at ArrayMethod structure we will see a couple of problems. (StringMethod structure is the same)
```rust
pub struct ArrayMethod<'a> {
method: &'a str,
variable: &'a str,
pattern: Pattern<'a>,
selection: Option<&'a str>,
}
```
1. The ArrayMethod structure has it's method name as one of it's argument. This serves no purpose in the code which is the method, other than to call it. The ArrayMethod should be redesigned so to either not need to know the name of the method (the parser calls the method as soon as it has read the method name), or ArrayMethod is a composit structure which consists of a structure which has a method name member and an argument's member which is a structure containing the arguments strictly needed to perform the method operation.
2. The ArrayMethod has one variable argument. Should it not be an array of variables? We can make more interesting methods if we were working with more than one argument.
3. The ArrayMethod has a pattern member. The pattern member is an enum which looks like this:
```rust
pub enum Pattern<'a> {
StringPattern(&'a str),
Whitespace,
}
```
I've tested what's put inside it and it seams like anything after the first argument in a in-shell method is put inside StringPattern as raw text. (Correct me if I'm wrong). Here is the rest of the variables I want my method to work with, in raw script source text. Now I have to manually parse them.
4. The ArrayMethod has a selection member. I don't know what that is, but I have a feeling that it should be pre-parsed away.
I suggest we go back to the drawing board with how method parsing should be done as it is a bit of a mess of how it's implemented.
My suggestion of a generic method structure is just to have an array of preparsed variables. No variable names needed to be de-referenced. No array expression needing to be created. The array should be typed and its contents must match the intended methods signature to be able to run.
Example:
```rust
pub enum Typed<'a> {
Array(&'a [str]),
String(&'a str),
}
pub struct MethodStruct<'a> {
variables: &'a [Typed]
}
```
The three culprit files are hidden in `src/lib/expansion/methods/`
BREAKING CHANGE: Changes in the datastructure, internal
performance: Might improve
usability: Adding methods for either ArrayMethod or StringMethod should be easier.
maintainability: Should remove code, making it easier to maintain.
reason: Refactoring for refactorings sake.
PS.
I've been rambling on about ArrayMember and StringMember. I just think there can be a lot of improvements to the method parsing in the files mentioned and that something should be done before even more methods needs to be written.
Oh and also testing tools should be rewritten. Why not test with the real ion shell instead of manually inserting variables into the shell heap with the DummyExpander?https://gitlab.redox-os.org/redox-os/ion/-/issues/1005Implement set -o pipefail2023-04-26T19:04:04ZAndrey BienkowskiImplement set -o pipefailfeat: I want `set -o pipefail`. #272 asked for both `errexit` and `pipefail`, but so far only `errexit` is implemented.
BREAKING CHANGE: it is possible to add `pipefail` without breaking existing scripts by defaulting it to off. In the ...feat: I want `set -o pipefail`. #272 asked for both `errexit` and `pipefail`, but so far only `errexit` is implemented.
BREAKING CHANGE: it is possible to add `pipefail` without breaking existing scripts by defaulting it to off. In the perfect world both `errexit` and `pipefail` would have been always on without any way to disable them (just like in all programming languages that have exceptions except for Visual Basic), but that might break existing scripts.
perf: impact
performance none
usability increase
maintainability increase
code: input
```
set -eo pipefail
echo 1
false | true
echo 2
```
expect: output
```
1
```
reason: I want my script to fail fast by default if an external command fails.
context:
Executing an external command can fail for a million different reasons. Here are some examples:
1. The external command is not installed
2. It encounters an internal bug and [fails fast and loud](https://en.wikipedia.org/wiki/Fail-fast)
3. It receives bad input due to a bug in my script
4. An admin kills it
5. The OOM Killer kills it https://linuxwheel.com/oom-killer-explained/
6. It tries to talk to another process possibly on a different machine and that fails for any reason
7. It uses a temporary file for internal buffering and /tmp is full
8. It tries to read or write a file and that fails for any reason
A robust script can not afford to ignore the possibility that any of the external commands it executes may fail. Often the right choice of action is to fail fast because the code that follows requires that the preceding commands executed successfully.
Unix-y pipelines have an issue:
```sh
foo | bar
```
If `foo` exits successfully `bar` sees EOF. If `foo` fails `bar` also sees EOF. `bar` has no way to know whether `foo` failed or not. In many cases `bar` will exit successfully even if `foo` fails. The script has to detect that and fail. This is why you often see `set -o errexit -o pipefail` in bash.
Unfortunately `set -o errexit` is [very broken](https://mywiki.wooledge.org/BashFAQ/105) in bash which is why I'm looking for a different shell.
<!--behavior of bash/dash/zsh/fish/oil-->https://gitlab.redox-os.org/redox-os/ion/-/issues/1004Ctrl-F jumps to end of line after history2021-06-24T21:01:48ZasvlnCtrl-F jumps to end of line after historybug: When using emacs keybindings, after using `Ctrl-P`/`Ctrl-N` to cycle through previously used commands, the first use of `Ctrl-F` jumps the cursor to the end of the line.
expect: Cursor should move one character to the right.
relat...bug: When using emacs keybindings, after using `Ctrl-P`/`Ctrl-N` to cycle through previously used commands, the first use of `Ctrl-F` jumps the cursor to the end of the line.
expect: Cursor should move one character to the right.
related: none
kernel: linux5.12
version: ion 1.0.0-alpha (x86_64-unknown-linux-gnu) / rev 9d76f020de899e05ae121ccfcbf7c85865b51276
interaction: none
context: This unexpected cursor jump occurs when first firing `Ctrl-F` after any new firing of `Ctrl-P` or `Ctrl-N`. The unexpected action persists even when cursor is moved with `Ctrl-A` or the arrow keys before firing `Ctrl-F`.
If `Ctrl-d` is called to delete a character before firing `Ctrl-F` then the bug does not occur and the cursor moves 1 space to the right as expected.https://gitlab.redox-os.org/redox-os/ion/-/issues/1003Add quiet flag to math command2021-06-17T13:47:39ZDavid PurdumAdd quiet flag to math commandfeat: For some automation tasks it would be nice to enter the interactive math mode without any welcome text. For this we could use a -q/--quiet flag to silence this (similar to R).
current
```
me:~# math
Ion's integrated calculator
Ty...feat: For some automation tasks it would be nice to enter the interactive math mode without any welcome text. For this we could use a -q/--quiet flag to silence this (similar to R).
current
```
me:~# math
Ion's integrated calculator
Type in expressions to have them evaluated.
Type "help" for help.
ion-math:
```
change
```
me:~# math -q
ion-math:
```https://gitlab.redox-os.org/redox-os/ion/-/issues/1002Cannot create empty slices when range start with the array length2021-05-31T23:26:46ZNils ErikssonCannot create empty slices when range start with the array lengthbug: It's not possible to create an empty slice when the start range starts with the arrays current length.
expect: That the slicing range 1.. would skip the first element and print nothing if the array only had one element.
related: n...bug: It's not possible to create an empty slice when the start range starts with the arrays current length.
expect: That the slicing range 1.. would skip the first element and print nothing if the array only had one element.
related: none
code: input
```
let foo=[bar]
echo @foo[0..]
echo @foo[0..0]
echo @foo[0..1]
echo @foo[1..]
echo @foo[1..1]
let foo=[bar baz]
echo @foo[0..]
echo @foo[0..0]
echo @foo[0..1]
echo @foo[1..]
echo @foo[1..1]
```
expect: output
```
bar
\n
bar
\n
\n
bar baz
\n
bar
baz
\n
```
actual: output
```
bar
\n
bar
ion: expansion error: invalid index
ion: expansion error: invalid index
bar baz
\n
bar
baz
\n
```
kernel: 5.12.4-arch1-2
version: ion 1.0.0-alpha (x86_64-unknown-linux-gnu)
rev 1170b84587bbad260a3ecac8e249a216cb1fd5e9
interaction: Run each command after the other
behavior of rust:
code:
```
fn main() {
let foo=vec!["bar"];
println!("{:?}", &foo[0..]);
println!("{:?}", &foo[0..0]);
println!("{:?}", &foo[0..1]);
println!("{:?}", &foo[1..]);
println!("{:?}", &foo[1..1]);
let foo=vec!["bar", "baz"];
println!("{:?}", &foo[0..]);
println!("{:?}", &foo[0..0]);
println!("{:?}", &foo[0..1]);
println!("{:?}", &foo[1..]);
println!("{:?}", &foo[1..1]);
}
```
output:
```
["bar"]
[]
["bar"]
[]
[]
["bar", "baz"]
[]
["bar"]
["baz"]
[]
```https://gitlab.redox-os.org/redox-os/ion/-/issues/998vi-mode: `e` does not work properly2021-08-31T08:28:31Zmseravallivi-mode: `e` does not work properlyin VI normal mode the `e` navigation does not work properly, it misses the last character.
How to reproduce:
```
[input from keyboard: ESC i l o r e m SPACE i p s u m]
$ lorem ipsum
[input from keyboard: ESC 0 d e]
$ m ipsum
```
Exp...in VI normal mode the `e` navigation does not work properly, it misses the last character.
How to reproduce:
```
[input from keyboard: ESC i l o r e m SPACE i p s u m]
$ lorem ipsum
[input from keyboard: ESC 0 d e]
$ m ipsum
```
Expected result ` ipsum`
This happens as well if I try to change the word:
```
[input from keyboard: ESC i l o r e m SPACE i p s u m]
$ lorem ipsum
[input from keyboard: ESC 0 c e h e l l o]
$ hellom ipsum
```
Expected result `hello ipsum`Ion Shell v1.0.0https://gitlab.redox-os.org/redox-os/ion/-/issues/994Scopes: can not change variables in function from matches, conditionals or loops2021-06-01T11:13:04Zmatu3baScopes: can not change variables in function from matches, conditionals or loopsbug: values in same function scope are not writable from matches, conditionals or loops.
Additionally, one can not write into variables of higher scope. We need to clarify the syntax for this.
expect: It should be possible to write valu...bug: values in same function scope are not writable from matches, conditionals or loops.
Additionally, one can not write into variables of higher scope. We need to clarify the syntax for this.
expect: It should be possible to write values in functions.
code: input
```sh
let x = 4
let y = 0
let z = 0
let a = 4
fn demo
if test 1 == 1
let x += 1
end
echo "x:" $x
if test $x != 5
echo "x broken"
end
for i in {0..=4}
let z += 1
end
echo "z:" $z
if test $z != 5
echo "z broken"
end
match $a
case 4; let a += 1
case _; echo "FAIL"
end
echo "a:" $a
if test $a != 5
echo "a broken"
end
#echo $y
#while test $y -lt 5
# let y += 1 # BROKEN: does not update y
# #echo $y
#end
#echo "y:" $y
#if test $y != 5
# echo "y broken"
#end
end
demo
echo $x
echo $y
echo $z
echo $a
```
actual: output
```txt
x: 4
x broken
z: 0
z broken
a: 4
a broken
4
0
0
4
```
expect: output
```txt
x: 5
y: 5
z: 5
a: 5
5
5
5
5
```
version: e033ca1c0c2ff8d1932137d2ed7354d003ad9cdc
interaction: noneIon Shell v1.0.0betamatu3bamatu3ba