There are some new nifty features in the OGN Logbook that you had not been told about. They are two rather nice and useful details some other fellow pilots already asked for or you might have desired.
Top Altitude During a Tow
For over two years you could have downloaded a .csv file in FlightOffice format that can also be opened in LibreOffice Calc or in any other spreadsheet tools. This file contains an summary of air traffic for each particular field for given day. The .csv file is available for download at top of the page in the page header as highlighted in the following screenshot.
Let’s assume you are from the Benalla airport, Victoria, AU (YBLA) and you have just started there as a tug pilot and local procedures require to record the maximal tow altitude for each flight. You can either note it down onto your kneepad every time upon tow release or alternatively check the appropriate column (col. H, MAX_ALT) in the .csv file where the logbook has recorded it for you (assuming you start the descend right after the glider release, the tow plane is equipped with well working OGN, transponder or other device and the OGN network coverage at your location is good).
The file structure is clear from the first header row and is I believe pretty self-explanatory.
Multiple flights in the Map View
This is a nice to have when you want to compare your friend’s flight with yours. Even when we agree to fly together in wing-to-wing formation, our paths usually split after a short while and many times it might be interesting which path the other one took. This feature is to visualise where our trails crossed without even knowing and where we joined on the cross-country flight under the same cloud again. For adding a flight just click on the large blue plus (+) symbol in the upper right corner of the map view.
Here you can search for flights originating from the same airport (by default) of choose some other location your friend might have departed from. Furthermore, the date can also be modified if you want for example to overlay two or more flights from various days in the same map. Just remember that the flight-path retention period is not infinite, hence your early spring shenanigans won’t be probably not be available in late summer. Enjoy! 🙂
In early February, when the sunny but still freezing weather allowed, I had been testing new firmware for the upcoming season 2023. The units under test were located on the roof of our lab building where new features had to be validated in field conditions. Especially helpful was the WT9 Dynamic in performing its repeated take-offs and landings at LKCM right behind the nearby peak.
For this firmware validations I conveniently use(d) the OGN Cube Control android app I developed some time ago for seamless upgrades without the necessity of removing the device from our gliders. Still on Monday afternoon it all worked flawlessly. On Tuesday (as I discovered months later) something happened. And finally on Wednesday morning when I wanted to apply the final firmware amendment I had just seen some Chinese symbols in the app! What the ****!? I keep my phone out of the endless Google update mayhem just to keep things consistent. Even the app was in the dev version (not installed from the Play) in order to retain full control over problems that may (and yes, they do) occur in the app. And the same issue appeared on all devices I know of at very similar time. So what was the trouble? Google had changed “something” in the very core of the Android API, pushed this improvement into all devices (even those with all updates explicitly disabled) and broke the app. And not only my app, many of them.
The solution seemed to be obvious – upgrade the android development toolkit, Flutter, Dart and the development workspace in general, clean, rebuild, deploy, done, right? Not at all! As of now the project is irreversibly broken and the app cannot be even build. It seem the entire app needs to be re-implemented for the new API from the very bottom up 😐
As many of you might have already guessed that is not that simple task for one summer weekend. Although the season is basically over the CUBEs still deserve firmware upgrade. The following is a step-by-step guide (for a bit more skillful readers) how to keep your CUBEs recent.
Prerequisites: you need a computer with (*buntu) linux (but any other distro will do just fine) and bluetooth capability. This can till be done easily in the hangar without removing your CUBE from the airplane.
Step #1: Run terminal/console app and install tools needed for communication between the scripts over bluetooth:
Step #3: Obtain the actual firmware file you need for your particular CUBE from a ognCubeReleases github repository. For the last step you need to know which MCU is inside your unit. To be über-certain you may want to open the box and see the type is in there. But in general, it should be like this:
OGN CUBE 2 -> STM32F103CBT6
OGN CUBE 3 -> STM32F103CBT6
OGN CUBE 3.5 -> STM32L152CC..
!!WARNING!! By flashing incorrect firmware “architecture” you might be unable to perform easy re-flashing again. It can still be done on site but it is a bit more elaborate..
The F103 (Cubes 2+3) have the aircraft type (glider/tow/uav) fixed in the bin file. Hence you want to choose the appropriate one. The CUBEs 3.5 can be configured and the fw comes as a glider by default. The configuration shall be retained during firmware upgrade but if you needed to change the type in the future I can tell you how someday later..
Step #4:Flash the firmware. The unit needs to be powered-on for at least 20s before executing the script below – not to be within the initial 10s-long bootloader mode (yet). The LED shall indicate it is in satellite acquisition mode (two consequent relatively short flashes –) or already transmitting (very short flashes twice a second).
The flashing process will reboot the unit automatically.
How do I know which version is the CUBE on? Connect to the unit using for example the Serial Bluetooth Terminal android app to see the prompted information. Here you can spot two dates – bootloader (comes right after power-on) and firmware version (right after bootloader yields control to the actual OGN code). The latter is what matters.
There has been some substantial movement under the logbook‘s hood. The python-based back-end regularly suffers performance problems during summer days with great soaring conditions which we all love but cause gray hair to the server’s administrator. In fact it would not be such a great deal, but when it happens it always bugs my mind.
The OGN beacon queue has been configured to ‘unlimited’ length for over two years now and the only way the problem is demonstrated are up to 12 hour long delays to process the piled-up data during such exquisite days while no precious data is lost. Considering the code fully utilizes all of the four powerful Intel Xeon Silver 4214 CPU @ 2.20GHz cores made me think there is something really rotten in the code. Moreover, there is a (a bit forced and already ongoing) plan to move the server onto a Raspberry Pi with significantly lower performance, something had to be done.
The decision process was clear – we need a significant speed-up for all necessary real-time operations. Python is (really, really) slow and there are only a few programming languages on top of the performance chart that could suit our needs. Based on my favourite information source Hackaday.com (and multiple others) the suitable options might be some of the following ones:
From top to bottom we may think of C, C++, Rust, Ada, Java and Pascal. C/C++ does not come into consideration as my experience with multi-threaded applications in C are not really positive (and I don’t really claim to be anywhere near to an experienced C developer even after all those years of developing the CUBEs 😉 ). Rust is really odd, Ada, Fortran (oh man, no intentions digging into those) and Java. I used to write a LOT of nice code in Java, but its memory footprint is a bit large for the little Pi. And you know what? Why not to try that Rust!!
Despite all the praising articles and blog-posts you can find around the programming pages and after almost a year of scrambling up the learning curve I dare to say Rust is the far most gory development experience I’ve ever encountered. Rewriting the core of OGN logbook took entire two full-time months of coding, full of despair and struggle with seriously weird and unfamiliar compilation problems, syntax and the whole approach to code in general. As people say “The experience doesn’t have to be good, it must be strong!”. In the end, after yet another month of testing, replacing many tricky .unwrap()s and polishing the details we have a working rust-based OGN logbook back-end. More experienced ‘Rustaceans’ may die in horror when looking on a rust code created by an OOP developer, but it is now as good as it gets.
Wait! This is still not end of the story! At this moment the logbook you use is still being driven by the python code. The reason is the amount of traffic as seen through the monitoring logs differs a bit. It might be as low as 5% in number of processed OGN beacons per minute, rising up to 10-20% during the peak hours. The APRS filters are identical (i.e. the entire world) and so are the rules for beacon filtering. Just the Rust book receives less beacons and, which is the most interesting part, also beacons for other airborne vehicles comparing to the Python client. Mostly there is and overlap but rather frequently there are planes that Rust can’t see while Python can. And vice versa. If any of you guys and gals had a clue why is this happening, please let me know! 🤓
Another improvement of the logbook coverage is extension of the airfields database by numerous small airstrips all over Canada. On Tim’s request a lot of data has been scraped, parsed and sorted resulting into 1638 new locations. The resulting data set consists now of notable 26903 airfields under OGN logbook‘s watch! 🙂
Although it might sound like a cliché, the component shortage made me to return to one nice and long adjourned project – the KRT2 radio adapter. The motivation for this one is obvious: since we were forced to exchange our perfectly working radios for another ones supporting the 8.33kHz channel spacing it was literally a pain to store anything in the stations’ memory. Every spring the same situation occurred in all hangars around the universe where infelicitous pilots were attempting to update the for some reason ever-changing frequencies into the by-the-devil-himself designed KRT2 radios. And then the rest of the season everyone refused to make any further amendments as the frustration is simply tremendous. Something had to be done! B-)
Luckily, the author(s) of the KRT2 radio let the option to upload frequencies from something called “KRT2 Manager WinXP” open, hence there is a chance for us to flash the channel configurations from outside – a “third party” app and not to suffer so badly anymore. The application is called KRT2 Bluetooth and thanks to a friend with an android developer account is available right from the PLAY store.
The adapter itself is a module that belongs between the radio and the connector which is already there. No sophisticated installation is required – you just smoothly plug it in and secure both sides with compatible latches.
After powering-up the radio, you pair your android phone with the bluetooth adapter, open up the app, configure your personal list of frequencies, select the appropriate bluetooth device and upload your configuration into the radio. Done. Finished.
Order of the stations in the list can be changed by dragging rows in the frequency-list table up and down. Multiple lists are beneficial especially when traveling around competitions frequently or friends desire to have a different set of frequencies. Complete lists can be exported into a csv file using the share button while import needs to be done by sending the file into the application (“Send to */*”).
I like to say one KRT2 Bluetooth module shall be present in every club so all the radios could be set-up easily at the beginning of the season and no hassle is necessary later on. I made several of these very handy units and you can get them right now! Just visit the Choose and Buy section 🙂
In the invent-build-publish chain I am somewhat neglecting the last step. In middle of January I had been asked whether and how well does the XCSoar navigation perform in combination with the OGN CUBEs. I actually always wanted to try that just for fun as we all already know the LK8000 provides a neat view of the other planes in CUBE’s radio range.
During the third week of January the temperatures were still freezingly low and the winds blew harshly but at least there finally came a rare sunny day. One of the remaining internal battery-powered CUBE3Bs was located at one side of LU’s testing facility’s roof while I climbed with the other CUBE3.5 depicted above to the other side – a bit higher and as far as it was possible. (Here you may also see the ‘new’ and experimental antenna which might deliver a better range.. but that is still unconfirmed.)
On the left you can spot the actual situation in the OGN viewer android application while on the right there is the appropriate configuration of the CUBE-XCSoar bluetooth pairing connection. Below the right picture shows the XCSoar map view with the other ‘glider’ as a little red static arrow. I have already misplaced the terminal screenshot but the NMEA sentences are being processed well as one would expect. And you can also compare the situation with the same view on the LK8000 screen in the the right screenshot. The zoom is at its maximum as the units were rather close to each other (well the roof is really not that large).
The screens below are older shots from the summer 2021 where LK8000 displays other gliders – one circling right above the LKKA runway (the green arrow) and two others eagerly expecting the tug plane (blue) to land and tow them to the air as well. The middle and right images are other possible modes of displaying similar situations in LK8000.
So the conclusion is YES, the OGN trackers work just fine with both XCSoar and LK8000 and can show you situation in the air when there are other planes equipped with appropriate gear around you. In the same way as the LX9000 with Flarm.. just a bit on the budget B-)
The holiday season finally freed my hands and let me to report on some new features implemented into both the OGN logbook web and engine. You’ve surely noticed the biggest and most valuable addition already: By following the little 🔍 symbol one can explore a track of a particular flight in detail!
There is one limitation, however. The flight-tracks can be accessed for only up to 24 hours after landing as there is a limitation imposed by the data retention policy. This (probably) prevents the logbook to keep the data any longer as I am not really certain how the handle interpretation of the “data redistribution”. Moreover retention of so much flight data for extended periods of time would quickly take up all the available server’s storage anyway, so it gets all dumped after that timeout automatically. It might be a nice to see our older flights but it could also cause harm when used inappropriately. Could somebody advise?
Second quite useful feature can be discovered when selecting a date in the header. For the date field to appear you need select an airfield or an airplane first.
Originally, this should have been a versatile date-range selection tool but apparently I am not as good web-designer as I needed or hoped to be. The possibility of selecting range of days, i.e. a weekend or an even entire week, had been here for some time now but is still not accessible with no bother from straight the UI. Nonetheless, you still have the opportunity to tweak the URL manually like for example this https://logbook.ibisek.com/loc/LKSU/2021-06-13/2021-06-16. Just be aware the maximum date-span is set to 14 days at most to ease the raspberry’s database load .. at least a bit.
Another big thing on the back-end, especially from the performance point-of-view, was to abandon the python threads in favour to the multiprocessing module. I might have known (but never realised that before) the python GIL (global interpreter lock) does not allow to run the python threads on multiple CPU cores in parallel (as one’s experience from other programming languages would suggest, right?). This was then a significant bottleneck causing long delays in beacons processing, typically when the season was at its peak. The processing capability topped at around 168k beacons/minute while all extra incoming beacons might have been queued for another (up to) 10 hours for being processed. Now the multiprocessing, on the other hand, would prevent us from stepping through the code in a debugger. For that reason, both of threads and multiprocessing are now used in the codebase depending on the mode of execution. You can get more of the details on GitHub.
The front page now shows traffic from all over the world. Initially, it was limited to the region you come from based on locale of your web-browser. The folks from Finland might not really been into flights in Italy, Swedes interested about traffic in Slovakia and Frogs about Britain. But in the end some us still might want to peek into what is going on in Chile, Namibia or New Zealand, particularly now in winter.
The airfields database has been extended to the incredible number of 25262 records by adding some extra Australian, NZ, south- and especially thousands of north-American airstrips. Thanks a lot, John! 🙂
Finally, besides the OGN, ICAO and FLARM the SafeSky beacons are now accounted for This traffic seems to be gaining some traction and we’ll see how that goes on in the next season. And also if it is of any use for us.
That would be all about the OGN logbook for now. News and updates about the CUBEs will come hopefully soon. Because there is also something going on! 🙂
It has been long three months since I’ve given some sign of life. There has been a lot going on all around and also new CUBEs caught a significant hold-up. And as the season is (again!) relentlessly passing by it was the highest time to take some time off and complete some open issues.
Most of the PCB assembly was done by Jan Weber in his workshop while I only populated the larger modules as it is easier to spot a problem and solve it before the more complicated and expensive components are soldered down. Here we encountered some troubles with the super capacitor backing-up the GPS as they refused to solder down properly and some of those even bursted into explosion during the heat-wave process.
Later on the most recent multi-protocol firmware was flashed into the new micro-controllers (I had written about earlier) and all CUBEs were quickly passing indoor tests one after another. During outdoor and in-flight testing some peculiar glitches were spotted but even those are gone and the code is now polished into almost perfection 🙂 Almost because there is still some ongoing work on oncoming features (which are disabled in this firmware release but will come later through the OGN CUBE Control update process at time allows.
The manuals were also updated and are already available in the downloads section. Finalisations are thus in full swing and the new CUBES 3.5 will set off to your airfields very soon! 🙂
First hints of forthcoming troubles with insufficient FLASH memory size of the STM2F103CB MCU’s started to peek out near the end of the last year’s season. With the ongoing development of new features my experimental CUBE was running short of program storage. With such expansion the codebase was about to hit the memory ceiling by the end of the year at the latest. Some thoughts were already thrown in the direction of yet another change in architecture design and which MCU shall all the existing code be migrated to. This is a story of the core MCU transition that took place during the last Xmas and spanning long into new years’ days.
As mentioned earlier in one of the previous posts the first OGN CUBE tracker prototype was born on then the all-possible-requirements-fulfilling 8-bit Atmel’s ATMEGA8-16PU RISC microcontroller. Five(!) years ago I managed to squeeze a working tracker code to 8 kB of FLASH and incredible 1024 bytes (BYTES!) of SRAM – a feat I cannot imagine today anymore. Very shortly, however, MORE memory was needed. Hence the first migration naturally went to ATMEGA328P – now widely known as the “Arduino” chip. Just imagine the unlimited possibilities of four times as much of FLASH a twice as much RAM! 32K Ought to be Enough for Anyone, right? 🙂 Well, not really but this tracker still proudly flies across the skies bearing the livery ‘PK‘.
The third migration was natural – I already had some experience with the STM32 ecosystem (F042 and F303 in particular), own stable set of SPL I/O libraries and also some hard time with HAL (which I strongly recommend to avoid). Hence it had to be a chip with a bright future, SPL support and some other reasonable parameters. After a long thought oscillation and numerous coin flips, the CUBE’s heart became the ubiquitous STM32F103CB IC, mainly due to its availability (at that time I even had no idea the other OGN trackers were based on the STM32F103C8). With 128kB of flash (C8 has only a half of it) and 32kB of RAM new way into bright future was paved. Are you now asking what was the other option? Wait for it..
The first Überraschung peeked out already after the first successful compilation – the binary size for equal code was twice as much in size on ARM as on the good old AVR MCU! It was a luck the -CB had been chosen instead of the -C8. But also during this new ‘ARM age’ the code was growing in size. Some optimisations had to be applied and features taken off – like for example the little-fs based SD card flight logging. The remaining FLASH was considerably quickly running out. But I am not indicating the ARM period is over 🙂
With new support of the other protocol B-) next huge leap was inevitable. Well, but in which direction? I needed the same footprint (LQFP48; the PCBs were already made) and ideally something from ST with SPL support (which is unfortunatelly left orphaned with their new MCUs), mainly because using HAL is a quick shortcut to HELL. Or .. could I possibly tease my imagination a little bit more?
Nordic’s NRF52 seems to be very popular with some folks. This bluetooth-equipped chip would also let me to throw away the standalone bluetooth module and save some power & space! But the ecosystem looks a bit odd to me. MSP430 from Texas Instruments? For some completely unknown reason I am being attracted to this micro but no, this would probably be a step in a wrong direction. A PIC? Oh, come on! (do you remember the Microchip vs. Atmel wars? B-) ) What about ESP8266? Not enough I/O. ESP32? FCC certified, a bit more I/O pins (still not enough), with on-chip/module Wi-Fi & bluetooth would be fantastic .. but its power consumption? Oh man! Something with RISC-V? There is nothing really useful and ‘production ready’ out there yet. And last but not least – what do you think about Teensy? .. it was getting wild! 🙂 But even after all these daydreams the CUBEs will have to remain in marriage with ST. It’s not a bad micro after all, really. So I went on a shopping spree..
One of the long-lasting temptations was to migrate to higher-end ST micros like the L4 (no SPL support ;( ) or F4 (SPL supported, with a floating-point unit, how sweet!). At the same time reduction in power was also one of my strong requirements. Unfortunately, they don’t make F4s in LQFP48 with 256kB of FLASH. Bugger! I had spend so many nights and even more spare time with the ST’s MCU Finder app (which has so much potential but they cannot keep it working reliably between updates; shame to you, ST!) until the final and only choice went to the same chip as two years ago when then eventually picking F103 – the STM32L152CC. It sports 256kB of FLASH, clocks up to 32 MHz (which is still ~ ok), 32kB RAM, some EEPROM to store user configuration and hopefully also some longer future 🙂
After an entire month (mostly December) of porting the code to the L152 (there really are some nasty differences and not only the the SPL; feel free to compare the block diagram of the MCU with the F103 in the datasheet) and hunting down the very last quirk in the I/O timings I can boldly announce the new MCU of choice for the oncoming OGN CUBE generation is the ST’s L152CC!
This write-up was supposed to come in February 2020 but some other, “more important causes” made me to reschedule a bit, well for almost a year. In late autumn 2019 I asked a friend with not only access to spectrum analyser but primarily with knowledge and expertise of using it to measure properties of five various antennas I acquired during that year mainly for experimental purposes. I have been carrying the charts presented below in my backpack for most of this time and now it is finally a good moment to share those measurement results with you.
This is the first and initial antenna which has been used with the CUBE ONE trackers. It took me some time to find a perfect match which would fit behind the canopy without drilling a hole into the hood. I had discovered this one on the Ampér 2018 eletronics fair that used to take place every April in the Brno exhibition centre. For long time I was not certain what the company name was. I just found out it was Sectron and the antenna type was AO-AGSM-TG09.
It was love at the first sight! Tiny dimensions, SMA connector, specifically designed for the 868MHz band and the ability to bend which made it so perfect. It works considerably well with metallic counterpoise which in CUBE ONE was located inside the box right under the PCB. Surprisingly, it performs rather well in air-to-ground traffic, despite the measurement result. Well, see for your self:
The M1 marker is around 730MHz while the frequency band we require (868MHz) is way out of the valley – find the M2 markers on the right. Also its configuration has a huge impact on its performance – the RED line is when the antenna is straight, light-blue when bent in 90 degrees and the bottom green one depicts its properties with a screwdriver in hand touching the outer SMA connector’s ring and making it almost well tuned. Nice but obviously for children to play as commented by Zdenek.
Why and how did I find this gemstone? Mainly because the CUBE TWO needed one and as I might have mentioned earlier I had just a fuzzy idea what company I purchased the first ones. It took some time but after endless nights spent with comparing and contrasting various types, sizes, brands and even colours I discovered the 2J010 from SOS Electronic .. which seems to be also out of stock at this moment .. 🙁
This story is still at its very beginning and I am already going to spoil the drama – this antenna performs the best from all here presented ones. It performs very nicely at 868MHz – see the markers M2 and M3 (-16dB or -23dB respectively). The M1 is set at 903 MHz. The bottom green line is again with a screwdriver touching the antenna and its influence is negligible. This antenna has the really best adjustment. It allows you to see other gliders at distance 5-7km all around you (well, this number hugely depends on mutual attitude and position of the gliders, of course).
Another specimen with a bendable joint. It looks like a wifi-piece but is not. Its performance is low. Attaching a screwdriver to the ground had not much of effect but significant at the joint. Even worse properties were observed when a hand was in close proximity to the antenna – rendering it rather useless. Next!
My personal favourite. It allowed the longest air-to-air distance I have encountered during the last gliding season – 23km! And that is only the farthest glider I have noticed in a moment of playing with the LK8000. And there might have been even longer distances! (it could be a good idea to add this information to the flight record..) Furthermore, it seems to to receive reliably other gliders at distance of 5-10km which makes it better even than the antenna #2. However, it is a bit bulky and kinda pain to fit it inside the tight cabin of LS-1. But with LS-8 it won’t be better 😉
Screwdriver in vicinity it had no influence whatsoever. Not even a hand around its root and all around the body. Its adjustment to the 868 MHz band is a bit worse but it has certainly better gain than #2. Marker M2 at 868 and M1 at 920 MHz.
Similarly to #3, the #4 has has been sourced from China which makes it with a nicer price tag (but who knows what will happen with shipping and taxes in 2021?) but with the sour uncertainty of repeatability of such purchase. Some other pieces I had bought were completely out of specs and violent manual adjustments had to be performed before throwing them to the garbage bin. This buy might have been pure luck but also a good source of nice antennas! Who knows? But I love it! 🙂
I spotted similar kind of antenna at the T-Cup 2019 in Káďa’s Cirrus. It seemed to be a good idea to use one (or two sector) antennas to scan the airspace in front of you with higher precision – not only to detect possible incursions but also to spot other contestant’s thermalling on the track before you. Her antenna was apparently custom-build piece of PCB (I had a photo.. somewhere) and I also wanted to explore this area a bit. For that reason I decided to obtain a completely different antena ‘architecture’ including a 2m long cable.
The measurement has shown it is adapted somewhere for the 900 MHz band. The cable, however, influences its properties significantly and even more do the dimensions and material of the body it is located on. Dead end.
The antenna #2 seems to have the best adjustment for the band we need for our OGN adventures while the #4 seems the provide the best air-to-air range. Both of them can provide pretty decent service in terms of range and reliability of data reception. Hence both of them shall and will be considered for the upcoming seasons 🙂 As the #2 is not available at this moment I might try to buy more samples of #4 and check if all of them have the same or similar properties like the single sample I have at hand. Still, buying stuff from China is a tricky business. Either way I will certainly share the outcomes once available! 🙂
As I mentioned earlier, what was supposed to be a one-weekend project has turned into four-months-long fine tuning effort and even now in November still not in excellent shape. In this post I would like to share the lows and highs which one can encounter when writing such on the first sight simple and straightforward application – the ognLogbook.
The first instances false take-off and landing detections were spotted when exercising spins and winch-launches at the LKKO airfield. Detection routines based mainly on ground speed (GS), initial thresholds values and minimal time of flight of 2 minutes (one cannot do a circle faster, right?) were failing when GS sunk down close to 0 km/h after a winch-based take-off, kept at 0 thru the climb and continued right above minimal GS limit during the circle. To make things even more tough those guys and gals also trained emergency landings across the runway making duration of one ‘circle’ flight just around 60 seconds.
Later that week I have noticed the second type of detection problem – GS close to 0 during spins with Blanik somewhere around LKVM. If you do more than one spin it already takes some time which had been evaluated as landing with conseqeunt immediate take-off. This trouble originated mainly from the vaguely defined touch-and-go detection routine.
Another madness had been detected at Krakowski Aeroklub in Poland as shown on the picture above. My hypothesis is they were testing their engine after an overhaul by taking off and landing immediately while still on the runway. The time, speed and altitude difference conditions were met and thus this was also detected as a (short) flight.
And there is even more crazy stories like that! 🙂
Ongoing detection problems made me to consider spawning yet another functionality – calculated flight altitude above ground level (AGL). This became quite tricky as all the available code examples did not work for me (a common situation), hence a bit hacking had to be used. You can get the EU terrain elevation data freely from the ESA’s Copernicus sattelite with 20m horizontal resolution (nice) in tile-files of total size around 20GiB. These had to be resized to tiles 500x500m to reduce its size to some 8MiB. Why? The initial workaround could be seen in calling a command line utility (gdallocationinfo) for every position (well, not every, just those near take-off and landing) by specifying latitude & longitude and parsing resulting output from the proceses’ output stream. This meant to load the (joined) tile-file from the hard-drive (SSD became a life-saver!) into memory every time the script was called and waiting for the process to finish. You can imagine the overhead! A month later I discovered the gdal binary version needs to fit exactly the python library (you need to downgrade in the system) and this enabled every received location’s altitude to be resolved from a more precise (200x200m) file of current size of 160MiB in real-time! Here is still one drawback, however: The elevation data covers only the territory of the European Union. The rest of the world is just dragons..
The most recent problems have arisen recently with the autumn wave season: The winds at FL195 blew so strongly the gliders came to have a negative GS! (greetings and congrats again to Hedlanda aeroclub at ESNC in Sweden! 🙂 ). This made the AGL calculation an indispensable step for every received position from the OGN network.
The initial ognLogbook‘s coverage was mere 300 kilometers around LKKA. Early on I started receiving inquiries if the area can be extended to cover this and that airfield so it has risen up to 1000 km with the center still at our hangar mainly due to the server performance issues. At that time the logbook was deployed on ‘CML6’, a machine based on remainder of an broken laptop with the Intel Core i7-3537U CPU @ 2.00GHz. This CPU hit its limit during a sweet summer day when the entire Europe went bananas and every available glider was pulled out of the hangars with a dream of at least 1000-kilometer-long flight 🙂 The last recorded beacon rate was 120000/min. The task queues were configured with ceiling of 1M (a million) records and the records that not fit into were brutally dropped.
But still this wasn’t really the biggest issue. The caches could have been configured as virtually limitless (well, only by total 8GB of RAM) and no data was dropped. Just then the data processing of such extent has taken until early morning hours .. two days later.
It was obvious that the 4 cores of the CPU (two physical) were not enough for five processing threads of the logbook application ( 1 – data ingestion, parsing and sorting, 2 – OGN beacon processor, 3 – FLARM beacon processor, 4 – ICAO beacon processor, 5 – maria db insertion queue and 6 – influx db insertion queue). The last one was only possible after migration to more powerful virtual server sporting four physical Intel Xeon Silver 4214 @2.20GHz cores allocated to just to our gorgeous ‘CML7’.
As long as the generously provided virtual machine is running we have a stable hive to process and further tinker with our OGN data. Currently, the entire flight record of every detected flying craft is stored into the influx for more detailed processing. Nonetheless, not for eternity – the retention period is configured for 7 days and then the data is discarded. The main reason is the tremendous amount of data: approximately 800 million records per week were stored at the and of August 2020.
There is still a lot of work that could be done – both in the realm of the data processing algorithms and also in the web interface (e.g. responsiveness, live data feed and more). Or rewriting into rust? If you were willing to pass a helping hand or can’t stand some issues you can see, you are warmly welcomed to contribute to the GitHub repository https://github.com/ibisek/ognLogbook. Any enhancements will surely be appreciated by everyone! 🙂