There has been some substantial movement under the logbook‘s hood. The python-based back-end regularly suffers performance problems during summer days with great soaring conditions which we all love but cause gray hair to the server’s administrator. In fact it would not be such a great deal, but when it happens it always bugs my mind.
The OGN beacon queue has been configured to ‘unlimited’ length for over two years now and the only way the problem is demonstrated are up to 12 hour long delays to process the piled-up data during such exquisite days while no precious data is lost. Considering the code fully utilizes all of the four powerful Intel Xeon Silver 4214 CPU @ 2.20GHz cores made me think there is something really rotten in the code. Moreover, there is a (a bit forced and already ongoing) plan to move the server onto a Raspberry Pi with significantly lower performance, something had to be done.
The decision process was clear – we need a significant speed-up for all necessary real-time operations. Python is (really, really) slow and there are only a few programming languages on top of the performance chart that could suit our needs. Based on my favourite information source Hackaday.com (and multiple others) the suitable options might be some of the following ones:
From top to bottom we may think of C, C++, Rust, Ada, Java and Pascal. C/C++ does not come into consideration as my experience with multi-threaded applications in C are not really positive (and I don’t really claim to be anywhere near to an experienced C developer even after all those years of developing the CUBEs 😉 ). Rust is really odd, Ada, Fortran (oh man, no intentions digging into those) and Java. I used to write a LOT of nice code in Java, but its memory footprint is a bit large for the little Pi. And you know what? Why not to try that Rust!!
Despite all the praising articles and blog-posts you can find around the programming pages and after almost a year of scrambling up the learning curve I dare to say Rust is the far most gory development experience I’ve ever encountered. Rewriting the core of OGN logbook took entire two full-time months of coding, full of despair and struggle with seriously weird and unfamiliar compilation problems, syntax and the whole approach to code in general. As people say “The experience doesn’t have to be good, it must be strong!”. In the end, after yet another month of testing, replacing many tricky .unwrap()s and polishing the details we have a working rust-based OGN logbook back-end. More experienced ‘Rustaceans’ may die in horror when looking on a rust code created by an OOP developer, but it is now as good as it gets.
Wait! This is still not end of the story! At this moment the logbook you use is still being driven by the python code. The reason is the amount of traffic as seen through the monitoring logs differs a bit. It might be as low as 5% in number of processed OGN beacons per minute, rising up to 10-20% during the peak hours. The APRS filters are identical (i.e. the entire world) and so are the rules for beacon filtering. Just the Rust book receives less beacons and, which is the most interesting part, also beacons for other airborne vehicles comparing to the Python client. Mostly there is and overlap but rather frequently there are planes that Rust can’t see while Python can. And vice versa. If any of you guys and gals had a clue why is this happening, please let me know! 🤓
Another improvement of the logbook coverage is extension of the airfields database by numerous small airstrips all over Canada. On Tim’s request a lot of data has been scraped, parsed and sorted resulting into 1638 new locations. The resulting data set consists now of notable 26903 airfields under OGN logbook‘s watch! 🙂