GPG

The GPG Key ID for Chris Punches (phanes/bagira on irc) is:

7A0E34E2EAA26356768D25A91F4B57A81ADDEFF5

It has been brought to my attention that someone on the Libera network is impersonating me on Libera.Chat using the nick phanes.

If there is any doubt about the authenticity of someone claiming to be me, the above credentials can verify my identity.

SILO GROUP Ambassador Program: Prawn

Introducing executive contributor Prawn.

He can be identified with the below key. He is hereby our newest participant in the SURRO Linux Ambassador program.

Ambassadors are allowed to represent the project as an agent to communities and chat networks, including IRC. This empowers them to set up new channels, take ownership of channels which we claim a right to the name of, and other important duties related to establishing a SURRO Linux community and official support channels.

Any questions about whether an ambassador is allowed to perform an action on behalf of the project can be directed towards ambassadors@silogroup.org.

His GPG key ID is 0x5C45C0FAF7C7BA6BBD2634081281CD445E54C98C

Advent Rex

Up until this point I’ve been building the prototype tool called Examplar while establishing requirements with the Foster build. For those of you just joining, Foster is the set of scripts and data that will generate the ISO or container for SURRO Linux.

I am pleased to announce that the Examplar name is now officially retired and will now be called Rex. It has been polished up, and can do some really neat things for large scale automation.

It’s source can be downloaded at the following URLs:

The primary repository is on our personal gitlab and the github repo is a downstream mirror so that we can get github users filing issues on its built-in tracker.

There are significant changes that needed to be made to get this tool from where it was when this project started to a usable product for this use case, so, it’s good to finally get it out of the way; I will now shift to just using it hopefully for the Foster builds I keep talking about so much while I knock out any bugs that come up from the issue tracker.

Really glad to get this piece in place, finally.

It is requested of anyone reading who can, to please review the code for potential improvements, file issues for potential problems (it is an alpha even though it’s 4th gen at this point), and let me know. Issues are filed on the github mirror for the time being.

-C

Mailing List Changes

After some consideration, I’ve decided to make some changes to the mailing list details for SURRO Linux.

While it’s not yet being used very much, I felt that using the term “greybeards” could potentially be excluding women from participating.

That, and, it turns out, as I learn more about how to facilitate an open source project, not all contributors contribute. I’ve been going about this all wrong — we should be trying to maximize the number of contributors to get that occasional pull request instead of trying to form a team of people who will do things regularly.

So, I’m disbanding the concept of a “greybeard” contributor and any associated roles, and now we just have “contributors” and “non-contributors”. If you don’t contribute, you’re a “non-contributor”.

The new mailing list is at:

http://mail.silogroup.org/mailman/listinfo/surro-linux_mail.silogroup.org

Contributors and non-contributors are welcome to sign up and help out.

-C

Official Gitlab Relocated

If you’re one of the reviewers, you’ve probably noticed that the source.surroindustries.com is throwing an error. That’s because it is now a CNAME record pointing to source.silogroup.org.

It was inevitable — the SURRO Industries name, while well intended, doesn’t reflect well from a branding perspective, and this really should be under the SILO GROUP name.

So, moving forward, source code repositories for all SILO GROUP owned projects not on github will exist at source.silogroup.org.

Significant Updates: A working Cross-Compiler

I’m excited to announce some significant progress on this effort since my last post.

After a couple years of frustration trying to figure out a way to design this process in a way that does not involve how LFS does it, which is problematic in so many ways between poor documentation in the binutils/gcc/etc docs, and the arduously inflexible documentation from the LFS project, I’ve decided to go a different way and test this out in a prototype called “Foster-B”.

https://github.com/SILO-GROUP/Foster-B

Intended to be a place where I can hack away without destroying my existing codebase with experimentation, I opted instead to litter the internet with various repositories of experimental sessions, in this case called Foster-B where I wanted to see if I could “outsource” the responsibility for maintaining the cross-compiler pieces to another entity without compromising SURRO’s “not based on another distro” requirement, like BUILDROOT would do.

At this point Foster-B has a working cross-compiler inside the container residing at /opt/crosstools.

The cross-compiler provider is the crosstool-ng project. It’s pretty clumsy but I got it working. And this should bring us up to the point in the process where LFS would be in about their chapter 6 using a method that is actually designed to be automated without potentially gutting the user’s host system.

This one part required a rather large investment of time and it’s not even done yet. I still have some cleanup to do, and I still need to use the cross-compiler to compile a chroot, which is actually a much easier task.

I think we’re through the “hump”, though, and I’m excited about days to come when I can be in that chroot building out the SURRO Linux proper.

Revisiting Foster

Foster’s main purpose is to be a proving ground for the components I’m building that will ultimately generate the SURRO Linux installer ISO.

I’ve been busy with work so it’s not been very active. I’m slowly revisiting my work with it. As most of you who do work with software can probably relate to, when you look at older code, even your own, sometimes it’s a little different than you remember it and a little more involved.

I’ve made this way harder than it needs to be.

So, I’ll go over some of the high level about what this is, how it works, and start a little cleaner as I wrap my head around how this should work.

Environment Preparation

It starts with a makefile.

The makefile in the Foster project is basically a wrapper for a set of shell scripts: one for make, and one for clean. As you might guess, make builds the Foster project. clean removes all the artifacts.

Docker

In the build script, you’ll see it’s mostly docker commands: I wrap a docker container around the whole build process.

The build script is just a kickoff point for docker pulling an image and spinning it up as a container. That container has a shared volume, which is the directory where the examplar project files are, as well as other files necessary to bootstrap into the environment I’m creating in the container.

Examplar

Examplar is, in this context, a build tool that is driving the whole build process.  It is also developed by me.

The docker image will then download the source code for Examplar, install any C++ compilation tools needed, such as the gcc compiler, and then build Examplar.

The image will then execute that freshly compiled Examplar instance against the Examplar project files from the shared volume mentioned earlier. The mechanism for this execution is the init.sh script which the docker container is told to run when it spins up. This script is located on the bootstrap shared volume as well. By the time this script is executed it means the instructions in the docker file have completed and the build is out of the environment preparation phase.

The init.sh script will execute the the examplar instance and pass on the return code from that examplar instance to its calling process to indicate success or failure of the build process.

The Examplar Project (Cycle 0)

Now that we’re in the part controlled by the Examplar project, examplar will execute units defined in its project files on the bootstrap shared volume. The order of execution of those units are defined in the plan file.

Environment Checks

It starts with environment checks and some primitive rectification attempts when it finds issues. When I say “environment” in this context I’m referring to the operating system of the running container. This includes present packages, their features and configuration, shell variables, and more.

Source Downloads

The source for the components to be compiled to create the new operating system are downloaded into a source directory.

CHROOT

Examplar will then create a directory at the root of the container OS filesystem. That directory will be the SYSROOT of the new operating system it is compiling, and will eventually be a CHROOT used to compile the rest of the pieces needed for a fully bootable system.

Cross-Compiler

Once this directory exists, examplar will begin compiling a cross-compiler from raw source to avoid sanitation issues created by upstream patches. That cross-compiler will be configured to output to the chroot. This step is problematic and is a work in progress.

The SYSROOT / Chroot Population

Examplar then begins compiling the bare, core parts of the system, such as the kernel and its dependencies, again dumping output into the CHROOT/SYSROOT directory. Once enough packages are present to compile librpm and rpm package management pieces, examplar will chroot into the newly populated sysroot and begin compiling pieces from the inside out. From there RPM packages are used.

Spring Cleaning and More

Foster Git Repo Given a Second Chance

Some time last year I accidentally pushed some source tarballs that created huge objects in the git repo for Foster.

Our newest contributor repaired it but we lost revision history in the process.  Since we’re still in pre-release and working out design still I’ve gone ahead and approved – we won’t miss those early pushes of just trying things to see if they work for some of the early R&D.

I’ve archived the original with the bloated repo objects in its revision history as those tend to be rather important later sometimes, but, we really don’t need them.

Foster is now Makefile driven and Completely Dockerized

Wrapping up the process in a docker container was pretty important for this project as we needed the build host to be abstracted away from the user’s operating system without requiring much setup from the user.

Using the dev’s host as a build host I don’t think is very safe because of all the things that can go wrong.  This was exemplified by a destroyed host after a typo in a log cleanup script last year.

We made it safe by wrapping the examplar build up in a docker container and image build process and orchestrating all of that with a parent makefile.

The usage details are in the Foster readme.

The only prior setup or dependencies Foster has now on the build host is the presence of docker and make on the system.  It should compile on Windows, Linux, or Apple operating systems now as a pleasant side effect.

We have a simple roadmap established for Foster

Foster will use the Fedora base docker image as its build host.  I’ve got an updated snapshot of Fedora so that folks don’t have to wait on crazy download times.  I’ve also implemented source caching for the libraries it builds into the chroot.  This will still require users to download it once by design.

Once the first bootable Foster ISO is created we’ll dockerize that image and use it as the base, and then go back and edit Foster to run on itself to remove our dependency on the Fedora base image.