Atuin v16 - New foundations, workspaces and more

Release notes for Atuin v16

Ellie Huxtable

Ellie Huxtable

— 11 min read

Announcing a new release of Atuin! v16 is out now.

This has been an interesting one. Two pretty core parts of the Atuin sync have
remained the same for the past couple of years, and performed better than I
ever anticipated. I'd always intended to replace the sync, but initially just
released something that worked "fine". Turns out, it worked far better than "fine"!

We've had issues though. In some rare circumstances, sync just won't complete.
In others, it requires atuin sync -f, which attempts to upload/download all
history again in case some has been missed. As we grow, it becomes increasingly
important that we revisit sync and ensure it is as reliable as possible - with
lots of users, "rare" circumstances happen more often.

In this release we're introducing the "key value store". This is a new feature,
built on top of new sync and new cryptography.

The reasoning behind this is two-fold - we'd like to extend Atuin's sync and
storage capabilities, but also build a sync protocol we can truly rely upon.

Conrad did an excellent job explaining the crypto we are using for this feature,
and I'll be following up with a similar post about the new sync protocol soon!

The goal here is to first prove that this crypto + sync are stable and
scalable, and then switch history sync over. Once done, we should be syncing
history much faster, with fewer errors, and with less data transferred. Your
data will also be kept safe with newer crypto algorithms, and we will have the
ability to upgrade and replace them as needed.

Community

Key value store

So first! Atuin v16 adds a kv store. This can be used for storing small
snippets of text and syncing them encrypted across machines. Env, small shell
scripts, whatever you can fit into 100KiB. After a sync, all kv pairs you set
will be available across machines.

Presently this should be considered "beta". Data loss is unlikely, and it has
been tested pretty thoroughly. However we don't make any promises for the
longer-term life or scalability of this, and you probably don't want to go
putting production secrets in there. There are many improvements we can make
to performance and user experience here.

We will also be using it to generally enhance Atuin (eg, synced config, host
aliases, etc). With v17, we hope to polish this off and release it fully.

Please do give it a try and let us know what you think!

# set foo = bar
atuin kv set -k foo bar

# kv supports namespaces
# set foo = bar in the "nsp" namespace
atuin kv set -n nsp -k foo baz

atuin kv get foo => prints "bar"

atuin kv get -n nsp foo => prints "baz"

# run sync to ensure you are up to date
atuin sync

Workspaces

Atuin now has workspaces! This is a pseudo filter-mode, which automatically
enables when you are in a git repository.

To enable workspaces, try

workspaces = true

in your Atuin config. With workspace filtering enabled, Atuin will filter for
commands executed in any directory within a git repo tree! Filtering modes can
still be cycled with ctrl-r.

Self hosted

This release runs database migrations on both the client and the server, though
nothing touching the history tables. No dependency versions have changed.

GitHub org

Atuin now lives in an org! The project has grown a lot since I first released
it back in 2021, and it's reached the point where an org makes more sense. Please ensure
you update any references to https://github.com/atuinsh/atuin!

ARM Builds

For a long time, Atuin has lacked aarch64 builds, which has been in demand
(especially for M1 mac). Cross compilation was never very reliable for us, but
GitHub actions does not offer runners for ARM.

For this release I bought an older M1 mac, which is now setup as a runner on
the main branch. All future releases will have ARM binaries available.

Growth stats

Atuin sync continues to grow! api.atuin.sh, at time of writing:

  • serves 2300 registered users
  • stores 27m (or 35GiB) of shell history
  • averages around 100,000 new lines of history per day
  • sees 40 signups per week
  • gets 1200 MAU, 1000 WAU, and 800 DAU

This data is gathered only from what we can infer from usage patterns, which
are tracked in order to prevent abuse. Your data is end-to-end encrypted and
can only be read by you.

Full changelog

New Contributors

Full Changelog: https://github.com/atuinsh/atuin/compare/v15.0.0...v16.0.0

Potential issues

If at any point after running v15 you ran main, there is a chance you will see the following error

Error: migration 20230619235421 was previously applied but is missing in the resolved migrations

Location:
    /atuin-client/src/record/sqlite_store.rs:53:9

This can be resolved by removing the records.db file inside the Atuin data directory.

On most systems

rm ~/.local/share/atuin/records.db*

During the development of v16, we updated a migration. This will only occur if
you ran a build from source, from main, after v15.

If you'd like to avoid issues like this in the future, I'd suggest avoiding
running main unless you are OK with the occasional breakage. Drop by our
Discord with any problems <3

Read more