I Wrote a Version Control System in Golang

Search for "Quick Links" to jump to Golang code samples and documentation.

I've tried a few different methods for managing dotfiles over the years. I was frustrated that they either required git and/or system links. I saw an opportunity to create a version control and file sharing system for single files with an emphasis on being user friendly.

Initially I spent some time learning how git worked. In summary: it models a directory of files as a tree of commits. A commit stores the changes between versions. This works well for a directory of related files where changes across multiple files should be bundled into the same commit.

For single unrelated files this isn't needed. If I only cared about a single file's history then I could make simpler system that avoided common pitfalls.

In summary I wrote a CLI tool and a web front end for sharing files between machines.

Introducing: dotfile and https://dotfilehub.com

Here are the features that I wanted:

• Save and restore past versions of files
• Push and pull files via remote server
• No merge conflicts
• Easily install files on a fresh or foreign system without dependencies
• Web application for finding other peoples files

Quick Links

• dotfile cli docs
• cli.go set up command line parsing with kingpin
• edit.go open a file in $EDITOR with os/exec
• local.go utility functions for interacting with file system
• storage.go manipulate dotfile tracking data
• dotfile.go compress and decompress data with zlib, regular expressions, sha1 hashes

The first step was designing a data structure for a tracked file. I decided to store the following:

• The file's path
• The current revision
• A list of commits
• An alias for the file

Commits are stored as list in chronological order. Each commit points to a full compressed revision of the file instead of the delta. This means commits are not dependent on each other, which avoids problems with merging unrelated histories. The downside is that is uses more disk space, though with small text files the difference is negligible.

I mapped aliases to files so that the user wouldn't have to memorize paths. When a file is initialized the user can choose to set an alias or take the default.

If ~/.config/nvim/init.vim was aliased to vim then the equivalent to the git command

git commit -am ~/.config/nvim/init.vim "<commit message>"

would be

dotfile commit vim

Much simpler!

Quick Links

• https://dotfilehub.com
• dotfilehub docs
• package db sqlite3 queries, database/sql code examples, database interfaces
• handlers.go generic and pluggable http handler with closures
• base.tmpl base template and dark theme support
• page.go go templates setup, FuncMap for injecting content into layout, flash errors, protected pages, user sessions
• routes.go http router with gorilla/mux
• api.go rest api

When it was time to implement push and pull I started on a web server.

I decided against using any major frameworks. At my work we use gin and gorm which I like, but for this project I wanted to explore the standard library further.

One of my goals was to make the UI simple and accessible enough that it would be possible to browse from basic browsers that don't support JavaScript. That way users could still find files in an environment without graphics using something like lynx. This led me to learn more about using semantic http and how server side rendering can be leveraged.

Another goal was to make files always available without dependencies. I added a special case to the file page to return plain text depending on the Accept header. So if you visit https://dotfilehub.com/knoebber/vim in a browser it will return HTML, but if you pass that url to CURL it will return the raw file.

This allows me to download my vimrc anywhere:

curl https://dotfilehub.com/knoebber/vim > ~/.vimrc

Database

I chose sqlite because of its excellent documentation, performance, and ease of use. Honestly I couldn't recommend it enough.

I wanted the database to use the same interface that the CLI uses on local file systems for modifying files. This would let it reuse code for operations like init, commit, checkout, diff, etc. I made interfaces Reverter, Commiter, Getter in package dotfile to accomplish this.

Next I designed the schema. The tables were users, reserved_usernames, sessions, files, temp_files, and commits.

I built some functions up to make common database tasks easier. The main component is the Executor interface:

// Executor is an interface for executing SQL.
type Executor interface {
        Exec(string, ...interface{}) (sql.Result, error)
        Query(string, ...interface{}) (*sql.Rows, error)
        QueryRow(string, ...interface{}) *sql.Row
}

Then I made every database function use a signature like:

// File retrieves a file record.
func File(e Executor, username string, alias string) (*FileRecord, error)

The advantage is that e can be both a plain database connection or a transaction.

Finally I made a function to generalize inserting records:

type inserter interface {
        insertStmt(Executor) (sql.Result, error)
}

type checker interface {
        check(Executor) error
}

func insert(e Executor, i inserter) (id int64, err error) {
        if err = validate.Struct(i); err != nil {
                log.Print(err)
                return 0, usererror.Invalid("Values are missing or improperly formatted.")
        }

        if c, ok := i.(checker); ok {
                if err := c.check(e); err != nil {
                        return 0, err
                }
        }

        res, err := i.insertStmt(e)
        if err != nil {
                return 0, err
        }

        id, err = res.LastInsertId()

        if err != nil {
                return 0, err
        }

        return id, nil
}

This validates the struct's data, optionally does a check, inserts the record, and returns the id of the new record.

Router

I considered building my own router, but after some research I decided that it would take too much time to implement the features that I wanted. I went with gorilla mux for routing and gorilla handlers for logging middleware. I like these because they use the net/http.HandlerFunc signature.

Templates

I made all the views with go templates. I used something close to a MVC style. First I made a struct for passing data to templates:

// Page renders pages and tracks request state.
// Exported fields/methods may be used within templates.
type Page struct {
        Title          string
        SuccessMessage string
        ErrorMessage   string
        Links          []Link
        Vars           map[string]string
        Data           map[string]interface{}

        Table        *db.HTMLTable
        Session      *db.UserSession
        templateName string
        htmlFile     string
        // Page access is restricted to their owners when true.
        protected bool
}

With this setup individual handlers look like:

func loadCommits(w http.ResponseWriter, r *http.Request, p *Page) (done bool) {
        alias := p.Vars["alias"]
        commits, err := db.CommitList(db.Connection, p.Vars["username"], alias, p.Timezone())
        if err != nil {
                return p.setError(w, err)
        }

        p.Data["commits"] = commits
        p.Title = "commits"
        return
}

Setting p.Data["commits"] = commits makes it available to the go template. In Ruby on Rails this might read: @commits = User.commit_list.

I'm happy with Dotfile overall. I find it to be useful for more than just dotfiles. It's sort of like pastebin with a CLI and versioning. I like being able to track and share any random file without headache. Here are the files that I've pushed: https://dotfilehub.com/knoebber

Obviously I have bias - lot's of people create their own system for managing their files, and I'm not suggesting that anyone move to this. It works for me, but I encourage everyone to find their own way.