Bcachefs, an introduction/exploration

Introduction & background information NOTE: This content is from an internal talk I gave, thus the reason it may read like a presentation So what is bcachefs? bcachefs is a next-generation copy-on-write (COW) filesystem (FS) that aims to provide features similar to Btrfs and ZFS, written by Kent Overstreet Copy-on-write (COW), with goal of performance being better than other COW filesystems Full checksums on everything Mult-device, replication , RAID, Caching, Compression, Encryption Sub-volumes, snapshots Scalable, 50+ TB tested Why the need for another FS?
Read more...

How-to: Writing a C shared library in rust

The ability to write a C shared library in rust has been around for some time and there is quite a bit of information about the subject available. Some examples: Exposing C and Rust APIs: some thoughts from librsvg Creating C/C++ APIs in Rust (site removed?) Rust Out Your C by Carol (Nichols || Goulding) (youtube video) Exporting a GObject C API from Rust code and using it from C, Python, JavaScript and others Rust Once, Run Everywhere All this information is great, but what I was looking for was a simple step-by-step example which also discussed memory handling and didn’t delve into the use of GObjects.
Read more...

Python, the perpetual time suck

I used to like Python. Like others I enjoy the productivity it offers and the vast and plentiful libraries that exist. However, over time that fondness has turned to loathing. The thing that should have been apparent to me long ago is that the Python folks don’t appear to care about end users. They seem to have lost touch with the fact that Python is very popular! Each and every time they make core language behavior changes, API changes, and deprecate things, a lot of code has to accommodate.
Read more...

DBUS Server side library wish list

Ideally it would be great if a DBUS server side library provided Fully implements the functionality needed for common interfaces (Properties, ObjectManager, Introspectable) in a sane and easy way and doesn’t require you to manually supply the interface XML. Allows you to register a number of objects simultaneously, so if you have circular references etc. This avoids race conditions on client. Ability to auto generate signals when object state changes and represent the state of the object separately for each interface on each object.
Read more...

How-to Stratis storage

Introduction Stratis (which includes stratisd as well as stratis-cli), provides ZFS/Btrfs-style features by integrating layers of existing technology: Linux’s devicemapper subsystem, and the XFS filesystem. The stratisd daemon manages collections of block devices, and exports a D-Bus API. The stratis-cli provides a command-line tool which itself uses the D-Bus API to communicate with stratisd. 1. Installation # dnf install stratisd stratis-cli 2. Start the service # systemctl start stratisd # systemctl enable stratisd Created symlink /etc/systemd/system/sysinit.
Read more...

Debugging gobject reference count problems

Debugging gobject reference leaks can be difficult, very difficult according to the official documentation. If you google this subject you will find numerous hits. A tool called RefDbg was created to address this specific need. It however appears to have lost effectiveness because (taken from the docs): Beginning with glib 2.6 the default build does not allow functions to be overridden using LD_PRELOAD (for performance reasons). This is the method that refdbg uses to intercept calls to the GObject reference count related functions and therefore refdbg will not work.
Read more...

Security considerations with github continuous integration

Continuous integration (CI) support in github is a very useful addition. Not only can you utilize existing services like Travis CI, you can utilize the github API and roll your own, which is exactly what we did for libStorageMgmt. LibStorageMgmt needs to run tests for hardware specific plugins, so we created our own tooling to hook up github and our hardware which is geographically located across the US. However, shortly after getting all this in place and working it became pretty obvious that we provided a nice attack vector…
Read more...

D-bus signaling performance

While working on lvm-dubstep the question was posed if D-bus could handle the number of changes that could happen in a short period of time, especially PropertiesChanged signals when a large number of logical volumes or physical volumes were present on the system (eg. 120K PVs and 10K+ LVs). To test this idea I put together a simple server and client which simply tries to send an arbitrary number of signals as fast as it possibly can.
Read more...

libtool library versioning ( -version-info ‘current[:revision[:age]]' )

The documentation on the gnu libtool manual states: The following explanation may help to understand the above rules a bit better: consider that there are three possible kinds of reactions from users of your library to changes in a shared library: Programs using the previous version may use the new version as drop-in replacement, and programs using the new version can also work with the previous one. In other words, no recompiling nor relinking is needed.
Read more...

Orchestrating Your Storage: libStorageMgmt

NOTES: Updated 4/2/2015 to reflect new project links and updated command line syntax Updated 5/20/2021 to reflect new IRC service Updated 7/18/2023 removed IRC channel Abstract This paper discusses some of the advanced features that can be used in modern storage subsystems to improve IT work flows. Being able to manage storage whether it be direct attached, storage area network (SAN) or networked file system is vital. The ability to manage different vendor solutions consistently using the same tools opens a new range of storage related solutions.
Read more...