diff --git a/content/news/rsoc-porting-tokio-to-redox.md b/content/news/rsoc-porting-tokio-1.md similarity index 97% rename from content/news/rsoc-porting-tokio-to-redox.md rename to content/news/rsoc-porting-tokio-1.md index e3495c9b36c5aba6d2e5a8b41e636f24d7b78598..97351b3e7343522919f6167c7d5f394b06efc352 100644 --- a/content/news/rsoc-porting-tokio-to-redox.md +++ b/content/news/rsoc-porting-tokio-1.md @@ -1,5 +1,5 @@ +++ -title = "RSoC: Porting tokio to redox" +title = "RSoC: Porting tokio to redox - week 1" author = "jD91mZM2" date = "2018-05-21T19:03:54+02:00" +++ @@ -68,6 +68,6 @@ and I was back to where I left off Day 3, but now with *much* better - and faste Now, what was the answer to why `tokio::spawn` wasn't working? Oh that was simple, my VM only has one CPU and it turns out tokio adjusts the number of worker threads to the amount of CPUs. -I mean, it probably should be work anyway. Guessing that could be a tokio bug. +I mean, it probably should work anyway. Guessing that could be a tokio bug. For now I just made it use at least two workers. This and a few other problems were patched, and here we are. diff --git a/content/news/rsoc-porting-tokio-2.md b/content/news/rsoc-porting-tokio-2.md new file mode 100644 index 0000000000000000000000000000000000000000..6c52005a00a09f439b7ebced2a10143a7f89e316 --- /dev/null +++ b/content/news/rsoc-porting-tokio-2.md @@ -0,0 +1,56 @@ ++++ +title = "RSoC: Porting tokio to redox - week 2" +author = "jD91mZM2" +date = "2018-05-30T12:27:25+02:00" ++++ + +# Porting tokio, update 2 + +I honestly never thought I'd be back with another post. +Almost all tokio examples worked, what else was there to do? +So I stopped writing daily updates to later compile to one summary (like this one), and went on with my life. +But programs have strange habits of breaking. + +Immediately after my last blog I was asked to try and port hyper. +With a few modifications, it worked! +Then I was asked to port reqwest, and that's where things started breaking. +Apparently the hyper I ported was a pre-release, meanwhile reqwest still used hyper 0.11.9. +And hyper 0.11.9 still used tokio-core. +You would've thought this was the end of it. "Too old tokio, can't continue". +But luckily I happened to have read somewhere that tokio-core is now a wrapper interface around the new tokio +so that old programs still work! + +Unlike hyper, reqwest didn't work out of the box. It received EOF too early when reading. +I was almost an hour in when I remembered that I got the same issue earlier! +Apparently tokio-core is not just a wrapper - it has some sort of copies of the tokio code. +What this means is that my old fixes to tokio were not available in tokio-core, so I had to backport those. + +Reqwest worked, but during larger transactions it still received EOF too early... randomly. +And it sometimes worked. And it sometimes only worked when I added debug prints... +After a few hours of nonstop cursing at my computer I noticed that because of redox's level-triggering model it +received old events and treated them as new, meaning one readable event could result in two reads, +where the second one reads 0 bytes - which is treated as EOF. +This was fixed by simply clearing another value as well when clearing readiness. + +The was still one last issue. Remember the `tokio::spawn` issue from the last blog? +Tokio failed to do multiple things simultaneously. +Last time I fixed that by increasing the number of workers, quickly dismissing it as a tokio bug. +Only one problem: tokio-core doesn't have a threadpool. +I really like this issue actually, it made me go back to an old issue and solve it properly :) +The fix was easy too, I suddenly remembered that calling `accept()` on a non-blocking listener on redox was blocking anyway. +And looking back at tokio's accept code I noticed that I simply forgot to clear readiness (since redox's event model is level-triggering). + +## Rounding up + +I was really annoyed by the impossible-to-debug random early EOF issue. +But I REALLY liked that the `tokio::spawn` issue wasn't upstream, +so I could go back and solve it properly myself. + +One sad thing is that almost all code I wrote will be able to be reverted once +redox changes to an edge-triggering model instead, +[which they most likely will](https://github.com/redox-os/netstack/issues/27). + +I've send pull requests to both [pkgutils](https://github.com/redox-os/pkgutils/pull/28) and [netutils](https://github.com/redox-os/netutils/pull/33) +that replaces the outdated hyper with the latest version of reqwest. + +I didn't expect to have to go back to my patched tokio code, but I'm happy I did :)