Albatross console out of memory

Summary

## Background

Albatross-console reads the console output from multiple unikernel tenders (solo5-hvt). This console output can be retrieved using albatross-client.

The console protocol is fairly simple: the unikernel invokes a PUTS hypercall, which sends arbitrary bytes of given length to the unikernel tender (host, typically solo5-hvt), which writes them to a file descriptor.

albatross_console reads this output, and assumes it will be newline delimited, and keeps at most 1024 lines.

## Problem description

This helps guard against unlimited memory usage from runaway/long running unikernels, however it doesn't guard against malicious (or buggy) unikernels.

The problem is this line in albatross_console:

``` let rec loop () = Lwt_io.read_line channel ```

Unfortunately Lwt_io.read_line doesn't take a parameter to limit the size of the line that is read, so it is very easy for a unikernel to exhaust the memory of albatross_console: all it needs to do is to write a lot of bytes without ever writing a newline.

Tested with the Debian packages, but confirmed the above code to be present in latest master too: ``` $ /usr/libexec/albatross/albatross-console --version version v2.3.0-9-g5b14787 protocol version 5 ```

## Impact

A typical attack will look like this in albatross_console's logs: ``` Jun 07 16:41:18 ubuntu22 albatross-console[13721]: albatross-console: [WARNING] disconnected Jun 07 16:41:42 ubuntu22 albatross-console[13721]: albatross-console: [ERROR] exception Unix.Unix_error(Unix.EBADF, "check_descriptor", "") while writing Jun 07 16:42:09 ubuntu22 albatross-console[13721]: albatross-console: [WARNING] disconnected Jun 07 16:44:07 ubuntu22 albatross-console[13721]: albatross-console: [ERROR] :mine error while reading Out of memory Jun 07 16:55:43 ubuntu22 albatross-console[13721]: albatross-console: [ERROR] exception Unix.Unix_error(Unix.EBADF, "check_descriptor", "") while writing Jun 07 16:57:20 ubuntu22 albatross-console[13721]: albatross-console: [ERROR] exception Unix.Unix_error(Unix.EPIPE, "send", "") while writing Jun 07 16:57:20 ubuntu22 albatross-console[13721]: albatross-console: [ERROR] exception Unix.Unix_error(Unix.EPIPE, "send", "") while writing ```

While the attack is happening albatross_console will also be very slow to react to the console of other unikernels, and will use increasing amounts of memory until an out of memory exception is raised.

Albatross_console stays running in my tests (obviously the bad unikernel will no longer have a functioning console, but that is to be expected).

However using so much memory can have an effect on other services running on the host.

## Workaround

Set MemoryMax=1G in the service file of albatross_console to limit the amount of memory a runaway console can use, so at least it fails sooner.

However in practice this just keeps albatross_console using 100% CPU without raising an Out of Memory exception, although it stays within 1G of memory.

There is no known workaround for the DoS attack, in theory Lwt should be switching promises when another one becomes runnable, but it doesn't provide fairness mechanisms.

## Solution

Use albatross in version 2.5.0 or above. Another solution is to apply the patch https://github.com/robur-coop/albatross/commit/d01805796ec710691a701c01ed3f0e4cd284a161 manually.

Binary builds of version 2.5.0 are available from https://builds.robur.coop

## Timeline

- 2025-06-07: issue discovered by Edwin Török as part of the HACKSAT25 challenge - 2025-06-07: initial email sent to hacksat@parsimoni.co - 2025-06-11: initial email sent to albatross maintainer - 2025-08-15: albatross 2.5.0 released with the patch