May 11 2015: Apache Spark, Dremelling hard drives, mixed mode UEFI and OwnCloud
Alice began by demonstrating using Apache Spark, an alternative to MapReduce with Hadoop, to analyse Leeds Road Traffic Accidents. Using the Scala shell, she read in the text file, created a Scala class, created an RDD (Resilient Distributed Dataset), cached it and then queried it to find the Pearson (linear) correlation between, for example, accidents with more than one casualty and the type of vehicle. It works faster because the data is held in memory and it is scalable. It can also query data held in other types of database including SQL. Since the latest version of Excel will link with Hadoop, it can be used to query Excel data.
Robert then demonstrated how to use a Dremel to make a faulty hard drive unreadable; it was pointed out that there are tools such as DBAN which can make a hard drive unreadable but also re-usable. However, where such tools will not work because the hard drive is apparently unreadable but still retains data, Robert’s approach is to be recommended. Faulty SSDs can generally be rendered unreadable by burning or dipping in acid which removes the surface.
Next he gave a short demonstration of Ethernet crimping and CAT5 wiring.
Matt then described the work he had been doing to enable the Linux kernel to support mixed mode UEFI where 32bit UEFI is used, for example, with the 64bit Intel Bay Trail chip. This involved translating an x86-64 ABI to i386. There are two phases: boot transition and runtime. During the latter, you have to lie to the firmware by telling it you are running in 32bit mode.
The kernel can tell how the bootloader was entered because a 32bit bootloader uses the legacy entry point (0) but a 64bit bootloader uses legacy + 512 bits. In order to enter 32bit mode, you have to map the physical to the virtual addresses, convert the 64bit addresses to 32bit, put them onto the stack and then reorganise the arguments. He warned that the firmware will return 32bit status codes that need to be converted to 64bit, that an unsigned long may not report its natural pointer size and that some UEFI runtime services, such as UpdateCapsule(), are not supported.
The following bootloaders will support mixed mode:
He pointed out that gummiboot is a UEFI application loader.
In response to questions, he said that there are no known bugs but it is, of course, very easy to attack most firmware.
He can be found on GitHub.
Martyn presented ‘OwnCloud: is degoogling hard?’ As a devop, he encourages devs to mess about with live services; that involves automation to put code live, enabling them to log in and encouraging them to write secure code that does not crash.
All the servers run Chef and they treat servers like cattle; if a server starts misbehaving, they shoot it in the head and rebuild it again by programming it to revert to a previously know state. Previously they had used Puppet.
So he decided to use a Puppet script to rebuild a virtual private server (VPS) and use the VPS to run OwnCloud. OwnCloud provides a repository for your own data, has a web interface, uses standards and allows you to run your own email server on a VPS. The scripts he used are on GitHub.
Install OwnCloud from your distro’s repos along with Vagrant; HTTPS is enabled and he uses StartSSL who provide free certificates. There is plenty of open source goodness; you can config on a per device basis; Android is supported very neatly.
Martyn also mentioned Keybase which provides public keys; however, that means they know your private key but they do allow you to use your own keys.
Using keys stops you using the web interface but not the CLI.
This led into a discussion of using time limited sub-keys.
Martyn noted that he had not been able to get a certificate for ‘degoogle’ because ‘Google’ is a registered trade mark.
Past Meetings