stevegattuso

I'm a programmer interested in urbanism 🏙️, sailing ⛵, traveling 🚄, and creating a more sustainable economy 🍃. This website is an always-in-progress repository for documenting my latest ideas and projects.

Finances & Budgeting

...with hledger and Tiller.

You Need A Budget was my go-to budgeting software for a long time. The software and the philosophy behind it have been really impactful in how I manage my finances and have helped me get through times where I've needed to be careful of my spending. I had no real compliants, however I found myself looking for a few things in an accounting solution that YNAB couldn't provide:

Enter plain text accounting. Specifically hledger. I don't have a particularly good reason for selecting this implementation of PTA over ledger or beancounter, but after ~9 months of use I've found it to be an excellent choice. I especially enjoy the thorough documentation on its website.

The setup I've arrived at largely mimics the big benefits of YNAB: I have envelope budgeting and automatic syncing with my banks. Additionally, the scripts I've built on top of this setup have allowed me to do things like monitor the value of investments over time, keep track of spending on a shared credit card with a partner, and even automatically charge friends for family plans.

Directory Structure

Everything in this setup lives in a single git-tracked directory which looks like this:

journals/
    all.dat
    2022.dat
    accounts.dat
    market.dat
args/
    spending.args
    budgets.args
    balance.args
    allocations.args
scripts/
    sync.py
    validate.py

journals/

This directory contains all of the journal files which hold transactions and their associated metadata. Each of these files are unified in all.dat, which looks something like this:

include accounts.dat
include market.dat
include 2022.dat

accounts.dat is likely optional depending on whether or not you want to use hledger's strict mode. tl;dr this mode will make it so you can't create transactions for any accounts which haven't been explicitly declared. I made the mistake of mispelling accounts multiple times and decided this was desirable, so this file contains stuff like:

commodity $

account assets:cash:checking
account liabilities:credit:acme-credit
...

account budgets:live:rent
account budgets:live:groceries
...

args/

HLedger has a lot of options. If you don't want to mess around with them each time you need to know if you can buy a pair of socks or not, argument files help immensely in accessing presets that give you the reports you need quickly. I have two of these so far:

budget.args shows me the current status of my envelope budgets using the balance command:

balance
budgets:*
-s            # Use strict mode
--tree        # Display it as a tree
--cumulative  # Show running total of budgets, not just this month's allocation

balance.args shows me the balance of all of my asset accounts (ie bank accounts) + liabilities (loans, credit cards, etc.):

balancesheet
-s            # Strict mode
-V            # Show everything in its current market value, in $
-5            # Limit the account tree depth to 5
-E            # Show empty
--tree        # Display it as a tree
assets:*      # The accounts to display...
liabilities:*

spending.args shows me a summary of my spending over any given timeperiod:

bal
--tree
--auto
expenses:*

scripts/

This directory contains various scripts for automating my daily budgeting tasks. sync.py handles syncing transactions with my banks (see below for details). check.py ensures my virtual account budgets sum up to match my real-world accounts and my ledger balances match up with my syncing service's balances. It runs on a git pre-commit hook to ensure I don't commit bad data to git history.

Automated syncing with banks

I really wanted to avoid using a SaaS company to get this done, but unfortunately banks in the United States are notoriously awful at allowing their customers to access their own data. Yes, you can technically download CSVs and QFX files from the banks' websites, but I didn't want to have to manually download and sync from n different bank's awful websites.

Instead, I found a service called Tiller which esentially acts as a bridge between Google Sheets and Yodlee. I looked into using Plaid or Yodlee directly, but they generally require a subscription and/or have overly complicated authentication schemes that I didn't feel like implementing. Instead, I decided to set everything up with Tiller and essentially use Google Sheets as my API (via gspread) to fetch new transactions using a Python script.

Unfortunately this script isn't in a state where I'd be comfortable sharing it just yet, but the general flow is this:

  1. Tiller automatically populates a spreadsheet with new transactions from my banks every morning.
  2. I run python3 scripts/sync.py manually when I have some time to reconcile transactions (usually once a day in the morning).
  3. The script pulls the full spreadsheet of transactions in via gspread and filters out any previously imported transactions.
  4. For each transaction, I run some custom logic that automatically categorizes it based on the payee and formats the transaction to flow into/out of the correct accounts.
  5. The script translates the transactions to plain-text accounting form and writes the resulting text to journals/2022.dat (or whatever the current year is).
  6. I go into journals/2022.dat with vim and make any remaining minor tweaks that may be necessary, ie categorizing a transaction from a payee that doesn't already have an automatically generated category.

One important detail: scripts/sync.py does not actually use hledger import at all. I found that hledger's importing logic wasn't quite what I needed to properly classify my transactions. This was especially true when I opened a shared credit card with my partner (see below for details on how this works).

All of this may sound complicated but once setup this workflow usually takes me around ~5 minutes or less per day. I've found it to be more than worthwhile for the peace of mind it brings me.

Envelope Budgeting

One of the key elements of YNAB's budgeting method is envelope budgeting. That is, each time you get a paycheck every dollar should be assigned to a budget. When you need to spend, you take money from the corresponding envelope with peace of mind, knowing that you've planned for the expenditure in advance.

Hledger's documentation contains a few different ways to set this up, but none of them really worked with the workflow I had. That is, many of them required dividing up the balance of a single account into various budgets, whereas I wanted something like YNAB which would allow me to divide up the total balance of all of my accounts into budgets.

Virtual postings get us close. They allow us to set the balance of accounts while bypassing the restrictions of double-entry accounting (ie money must flow from somewhere to somewhere else). For example, each time I get a paycheck I'll do something like this:

2022-01-01 * Paycheck
    income:my-company:salary    $-1000
    assets:cash:bank:checking   $1000
    (budgets:unallocated)       $1000

Now I have $1000 in an "unallocated" budget, similar to YNAB's "ready to assign" balance. In a separate transaction, I'll handle assigning those dollars a job:

2022-01-01 * budgeting
    [budgets:live:rent]       $500
    [budgets:live:groceries]  $250
    [budgets:fun:travel]      $250
    [budgets:unallocated]

Notice the use of brackets instead of parenthesis! This transaction is using balanced virtual postings, which ensures that all of the virtual postings add up to $0 (like in normal double-entry accounting). When we're writing inter-budget transactions this is desirable, as we want to ensure we're not moving around money we don't have.

Now when I need to spend money, I'll have a transaction like this:

2022-01-02 * Rent
    assets:cash:bank:checking   $-500
    (budgets:live:rent)         $-500
    expenses:live:rent          $500

Notice that expenses:live:rent aligns with budgets:live:rent. This is very intentional, as it allows me to easily set up two matching reports: one for how much money I have available in each budget and another for how much money I've spent from each budget.

The main issue with this technique is that you're essentially maintaining two sources of truth for how much money you have available. You have the total of your account balances (eg assets:cash:*), and the total of your budgets (eg budgets:*). These two need to stay exactly in sync in order for you to be able to trust the numbers you see in your budget report.

If you trust yourself, you can just periodically look at these two numbers and make sure they add up. I decided that I emphatically do not trust myself, and therefore have a script, scripts/check.py, which adds up the totals from assets:cash:* and compares them with the total of my budgets:* virtual accounts. If these two line up, the script will pass. If it does not, the script fails and notifies me how much the difference is. If I'm feeling fiesty I can go through and manually try to reconcile the two properly, but many times I get lazy and just do something like this:

2022-01-03 * budgeting
    ; ugh I don't know why but my budget is $3.54 below my account bal. sorry
    ; travel budget, but you need to make a sacrifice to cover up my mistakes.
    (budgets:fun:travel)  $-3.54

Don't look at me that way. We've all gotten lazy and done lazy things like this.

Sharing expenses with a partner

My partner and I have a credit card that we use for shared expenses like groceries. This poses a slight problem, as I like to keep very meticulous budgets and they do not. Thankfully hledger is flexible enough to handle just about any accounting setup, and I've managed to settle on a workflow that allows me to continue being meticulous without requiring my partner to think or know about the craziness in any capacity.

In my scripts/sync.py I have some logic that will take all transactions from our shared credit card and format them like so:

2022-02-01 * Grocery Store  ; shared:
    ; total                              $-22.50
    liabilities:credit:shared:steve      $-11.25
    liabilities:credit:shared:partner    $-11.25
    (budgets:live:groceries)             $-11.25
    expenses:partner                     $11.25
    expenses:live:groceries              $11.25

Voilà! The expense has been divided up into two parts, one within the realm of my crazy budgeting system and another in my partner's own expense category that I can ignore from my reports. Once we receive the statement from our credit card company we can easily know who owes how much and divide up the bill accordingly.

In addition, the shared: tag allows me to easily run reports on our shared spending habits, which I can then export to my partner's preferred format: a simple csv.

Is this overcomplicated? Yes. Was it fun to build? Also yes. Does it drive my partner crazy? Maybe...