stevegattuso

I'm a programmer who enjoys learning/talking about cities 🌆, energy ⚡️, (vegan) cooking 🍃, biking 🚲, traveling 🚞, and creating a more sustainable economy 🏴‍☠️. This website is an always-in-progress repository for documenting my latest ideas and projects.

Switching from YNAB to hledger

You Need A Budget has been my go-to budgeting software for a long time. The software and the philosophy behind it have been really helpful in helping me navigate times of tight finances.

I don’t have anything bad to say about YNAB, but I’ve found myself with some extra time and wanted to explore alternatives. Ledger has been on my list of things to try for a long time and I decided to take the plunge, though for some reason I ended up with hledger instead of the original implementation. I don’t have a great reason for this choice, and the two appear to be largely compatible with each other so I suppose it doesn’t matter much.

NOTE: I’ve been making this switch largely out of curiosity and am still not sure if I’ll stick with hledger over YNAB. I’m documenting this out for the benefit of anyone else in a similar situation considering making the switch- hopefully you can build upon my learnings.

This page is a work in progress- I’ll try to keep it updated as I either continue or stop myself from falling down this seemingly endless rabbit hole.

Why do this?

A few of the reasons ledger has interested me:

  1. YNAB is a SaaS company and I generally have low trust for these kinds of entities when it comes to stability. That being said, YNAB has been around for a long time and when I went to see if they were VC backed I ended up on their careers page which was impressively generous towards employees. Given that they do profit sharing, I’m guessing they’re actually owned by their founders which makes me think they merit more trust than the average tech startup. +1 for YNAB!
  2. HLedger can track multiple commodities whereas YNAB can only track dollars. This is useful if you deal with multiple currencies or if you want to track assets like stocks, mutual funds, etc.
  3. HLedger is open source and seems to have an active community behind it.
  4. The simplicity of Plain Text Accounting is appealing. I like to keep my data in formats that are portable and long-lasting. Why not apply that to my financial data?
  5. Using a command line utility for accounting opens up infinite possibilities of scripting and automation- even though it’s accounting (which is kind of boring) I like the idea of being able to automate processes that are specific to my workflows rather than have to adopt somebody else’s.

My setup

I found learning the basics of ledger to be relatively straightforward. The getting started guide on hledger’s website is very approachable and the user manual + cookbook page are pretty comprehensive for everything else I needed.

Basic structure

After a bit of fidgeting I ended up with the following directory/file structure.

ledger/
  journals/
    all.dat           # This file includes all of the rest, and is where LEDGER_FILE points to
    accounts.dat      # Used to define accounts for strict mode
    budgeting.dat     # Transactions which modify my envelope budgets
    markets.dat       # Stores latest prices for commodities/currencies
    checking.dat      # The rest are for transactions associated with each account
    credit_card.dat
    savings.dat
    investments.dat
  scripts/
    sync.py           # Auto-imports transactions from Tiller and Venmo
    check.py          # Run sanity checks
    markets.py        # Syncs latest market prices
  args/
    budget.args       # Displays current envelope budget balances
    balance.args      # Displays balances of all accounts
  csvs/
    default.csv.rules # This file is included by each of the .rules files below
    checking.csv
    checking.csv.rules
    savings.csv
    savings.csv.rules
    credit_card.csv
    credit_card.csv.rules

Journals

Notable amongst the journal/ directory is all.dat. This file doesn’t actually contain any transaction data, rather, it includes in each of its sibling files so that there is a singular path that the LEDGER_FILE env variable can point at which contains all of my transaction data. It looks something like this:

include accounts.dat
include budgeting.dat
include markets.dat
include checking.dat
...

accounts.dat is likely optional depending on whether or not you want to use hledger’s strict mode. tl;dr this mode will make it so you can’t create transactions for any accounts which haven’t been explicitly declared. I made the mistake of mispelling accounts multiple times and decided this was desirable, so this file contains stuff like:

commodity $

account assets:cash:checking
account liabilities:credit:credit_card
...

account budgets:live:rent
account budgets:live:groceries
...

Args

HLedger has a lot of options. If you don’t want to mess around with them each time you need to know if you can buy a pair of socks or not, argument files help immensely in accessing presets that give you the reports you need quickly. I have two of these so far:

budget.args shows me the current status of my envelope budgets using the balance command:

balance
budgets:*
-s            # Use strict mode
--tree        # Display it as a tree
--cumulative  # Show running total of budgets, not just this month's allocation

balance.args shows me the balance of all of my asset accounts (ie bank accounts) + liabilities (loans, credit cards, etc.):

balancesheet
-s            # Strict mode
-V            # Show everything in its current market value, in $
-5            # Limit the account tree depth to 5
-E            # Show empty
--tree        # Display it as a tree
assets:*      # The accounts to display...
liabilities:*

Budgeting

Obviously a large draw to YNAB is their implementation of the envelope method of budgeting. I found this guide linked from the Cookbooks page of the hledger manual which detailed how to go about implementing envelopes using sub-accounts. After giving it a shot, I found a major issue with this method is that your budgets must be sub-accounts of a physical account (ie your checking account) and can’t be mixed among them. This means that if you were to have two checking accounts for spending, your budgets would have to be split amongst the two rather than having a pool of “cash to be allocated” that can be put in whatever budgets you’d like.

This may work for some people with a single bank account but for my particular setup I found it to be limiting. I ended up venturing into my own territory by using virtual postings for my envelope budgets. This setup more closely imitates YNAB by allowing budgets to have funding that spans multiple physical accounts.

To set it up, I first made a few transactions with assertions to place the amount of money I had available in cash into an unallocated budget. For this example we’ll assume we have $1000 total across all of our physical accounts:

2022-02-01 * Initial budget transfer
    assets:cash           ==* $1000  ; Ensure we're correct in assuming we have $1000 available
    (budgets:unallocated)     $1000
    [budgets]             ==* $1000

Next we’ll take the money in the unallocated budget and assign each dollar a job:

2022-02-01 * Budget allocations
    [budgets:live:rent]           $500.00
    [budgets:live:utilities]      $100.00
    [budgets:live:groceries]      $150.00
    [budgets:fun:restaurants]     $250.00
    [budgets:unallocated]

Once we’ve done that we can see the current value of our budgets:

$ hledger -f ./test.dat bal budgets --tree
            $1000.00  budgets
             $250.00    fun:restaurants
             $750.00    live
             $150.00      groceries
             $500.00      rent
             $100.00      utilities
--------------------
            $1000.00

Whenever I make a purchase it’ll look something like this

2022-02-01 * Some Vendor
    expenses:fun:restaurants     $10.00
    assets:cash:checking        -$10.00
    (budgets:fun:restaurants)   -$10.00

Note that the expenses:* sub accounts mirror the structure of the budget accounts exactly. This allows you to pretty easily see how much you have left and how much you’ve spent from each budget:

$ hledger -f ./test.dat bal budgets --tree
             $990.00  budgets
             $240.00    fun:restaurants
             $750.00    live
             $150.00      groceries
             $500.00      rent
             $100.00      utilities
--------------------
             $990.00

$ hledger -f ./test.dat bal expenses --tree
              $10.00  expenses:fun:restaurants
--------------------
              $10.00

The one catch to this method is that it is critically importand that the amount of cash you have in your physical accounts exactly matches the amount of cash you have allocated in your budgets. That is, the sum of cash:* must be equal to the sum of budgets:*. I’m still not exactly sure how to do this in Ledger. It seems like there should be a way to do a balance assertion across acounts but I’ve yet to figure it out. In the interim, I’ve added a simple function in my check.py script which ensures this is true each time I run it (usually after reconciling):

import subprocess
import pandas as pd

# This array is effectively same as YNAB's designation of an "on-budget"
# account.
BUDGETED_ACCOUNTS = [
    'assets:cash:account1', 'assets:cash:account2', 'liabilities:credit:card',
]

print('--- Verifying budget totals ---')
# Fetch the amount of cash we have, stick it into a dataframe, and extract the
# total
cash = subprocess.check_output([
    'hledger', 'bal', *BUDGETED_ACCOUNTS, '-O', 'csv',
]).decode('utf-8')
cash = pd\
    .read_csv(io.StringIO(cash))\
    .set_index('account')
cash_total = cash.iat[-1, 0].replace('$', '').replace(',', '')

# Same process but for the budget accounts
budgets = subprocess.check_output([
    'hledger', 'bal', 'budgets:*', '-O', 'csv',
]).decode('utf-8')
budgets = pd\
    .read_csv(io.StringIO(budgets))\
    .set_index('account')
budget_total = budgets.iat[-1, 0].replace('$', '').replace(',', '')

# Verify that all looks good...
if budget_total == cash_total:
    print('All good!')
else:
    print('!! Budget total does not match cash total:')
    print(f"\tACT: {cash_total}")
    print(f"\tBUD: {budget_total}")
    print(f"\t     {round(float(cash_total) - float(budget_total), 2)}")

Auto-import (v1 - Tiller)

The biggest hassle was setting up auto-importing from banks. This is a feature I really value from YNAB, as I don’t want to have to manually enter in transactions and I also don’t want to constantly be exporting CSVs from each financial account’s website in order to keep things up to date. I noticed that the library author had a page detailing his setup which mentions using Tiller for syncing. This seemed promising so I ended up going with that.

Tiller is a service that allows you to log into all of you bank accounts and automatically export the transaction data to a Google Sheet. This is great until you realize that Tiller has no API, so in order for the data to go from that Google Sheet to your Ledger database, you’ll need to either export the sheet as a CSV or use the Google API. For the sake of automation, I opted for the latter, using the gsheets Python library to fetch the spreadsheet into a simple syncing script I wrote.

The code looks more or less like this:

import pandas as pd
import gspread
import os

# Only sync transactions that have occurred after the day we started ledger
# (ie set our initial balances)
INITIAL_LOAD_DATE = '2022-02-16'

# Tiller will likely use account names that you don't want to transfer into
# ledger. Correct for that here:
ACCOUNT_MAP = {
    'Account ID in Tiller sheet': 'my_account_id',
}

# Generate some paths relative to our ledger root directory. Note that
# LEDGER_FILE points to a path within the journals/ directory
CSV_ROOT = os.path.join(os.path.dirname(os.environ['LEDGER_FILE']), '..', 'csvs')
JRN_ROOT = os.path.join(os.path.dirname(os.environ['LEDGER_FILE']), '..', 'journals')

# Fetch the current Tiller sheet and convert it to a pandas DataFrame (this
# is likely overkill but I'm more comfortable in pandas so I went with it)
gc = gspread.service_account()
sheet = gc.open('tiller')
worksheet = sheet.worksheet('Transactions')
df = pd.DataFrame(worksheet.get_all_records())

# Filter out any transactions that happened before our initial load date
df['Date'] = pd.to_datetime(df['Date'])
df = df.loc[df['Date'] >= pd.Timestamp(INITIAL_LOAD_DATE), :]

# Filter out any unwanted columns
df = df[['Date', 'Description', 'Amount', 'Account']]

# Loop through each account, saving its transactions into its own CSV file and
# then running hledger in order to display a preview of any new transactions
# that were found.
for account in df['Account'].unique():
    print(f"---- {account} ----")

    account_txns = df.loc[df['Account'] == account, :]
    csv_path = os.path.join(CSV_ROOT, f"{ACCOUNT_MAP[account]}.csv")
    print(f"Writing {len(df)} lines")
    account_txns.to_csv(csv_path, index=False)

    print('Preview of transactions to be imported:')
    dat_path = os.path.join(JRN_ROOT, f"{ACCOUNT_MAP[account]}.dat")
    subprocess.run([
        'hledger', 'import', csv_path, '--dry-run', '-f', dat_path,
    ])

After running this script each account’s CSVs will be populated with the latest transaction data. Assuming the import preview looks good, I’ll then run…

hledger import -f ./journals/[account].dat ./csvs/[account].csv

…to import the transactions into the journal files then go in and manually edit any accounts/transaction descriptions as needed.

Once any manual touchups are done, I’ll run check.py to ensure all of the local account balances match up with the remote balances on Tiller. So far this has been a pretty reliable process, but we’ll see how it fares as I encounter new financial situations.