April 8 2025 Incus, SciPy, K-nearest, k-means, spreadsheets vs databases, FAT32
Bernie reported that Canonical has taken all development of LXD in house; so a number of the developers have created a fork called Incus. There are lots of images available on the Incus image server including ones for Ubuntu and Debian trixie which is currently in testing. So it will eventually be available as an APT. Bernie has added an Incus repository to the Raspberry Pi and created an Ubuntu and a Debian container on the Raspberry Pi.
Incus commands are mostly the same as for LXD. It can take snapshots and you can create multiple containers from a single image.
Incus offers both containers and virtual machines.
Darren mentioned the Python features for scientific exploration, in particular that most processors have branch prediction and that it is important to optimise code for branch prediction. He had also been looking at the algorithms to deal with recurring fractions and how they calculated boundaries of error in order to work out the shortest way of dealing with recurring fractions. Bernie noted that there are lots of Python libraries dedicating to scientific subjects, such as SciPy, while John recalled dealing with the problem in dBASE II in the 1980s by adding something like 0.0001 in the code at the end of a calculation to get dBASE II to return the correct whole number.
Darren added that APL has a routine to work out what is the most accurate number while LISP uses vulgar fractions and then simplifies the output. To do this you need special arithmetic modules in LISP.
John Walker asked whether you can you use Jitsi anywhere and John said that you can; however, to use Jitsi’s own server, you have to sign in. But, there are free servers, such as the one we are using, where you do not have to sign in.
Darren’s next OU subject is data science classifications using the K-nearest and k-means algorithms to classify new data where the data has lots of data with points relatively defined but where the groups are not well defined; the hard bit is defining your higher dimensional space. [The SciPy and Scikit-learn libraries have modules for k-means.]
A starting point can be Pythagorean metrics using cosine similarity to plot the cosine distance between the vector to one set of data and the vector to the new data and use the angle to determine the classification.
John commented that it looked like the sort of technique used with to identify new viruses and Darren said that the technique had been used in 1930s with plants and calculators able to calculate to three decimal places! The Towards Data Science website has some useful material on it.
This discussion led into a short discussion on how some scientific discoveries such as the coronavirus, the depletion of the ozone layer and pulsars were totally unexpected. It was only because the pulsars came at very regular intervals that people realised that they must come from outside the solar system.
Darren then shared an experience of working for a company which tried to do everything with Excel and he had realised the limitations of Excel because everything had to be done in RAM and so there were limitations when he began to create pivot tables. So the company had suggested that he use Access where data is stored in permanent storage. John commented that, for data manipulation, spreadsheets are also limited by the maximum ‘field’ or ‘column’ length. Unfortunately, a lot of people think they can use Excel to do things when they would be far better using a dedicated program like an accounting or database program.
Darren added that he realised that FAT32 has a limitation of 4GB file size; John commented that he had hit this problem with a DVD player which will accept USB input as an alternative to a DVD but only accepts FAT32 format USB sticks or drives. If Darren does not need the USB stick to be FAT32, he can reformat it to NTFS. John has an old dual boot computer to which he has added a 64GB NTFS formatted USB stick which is accessible to both Windows and Linux.
Finally, Darren mentioned that he was thinking of resuscitating an old computer with a small amount of data on it. John suggested that he download SystemRescue and copy it to a USB stick. It should then be possible to boot it on the computer (in some cases after changing the boot sequence to ‘Legacy mode’), mount an empty USB stick and copy the data he wants onto the USB stick. He can then do a fresh re-install on the computer before restoring the data now on the USB to it. John added that once he had come across a computer which would not accept a USB boot and he had had to plug in a CD drive with SystemRescue on that.
By this time we had run out of computing topics and the discussion turned to biorhythms and some of the mathematics involved in them.
Past Meetings