nup

Build Status

App Engine app for streaming a music collection.

Overview

light mode screenshot dark mode screenshot

This repository contains a server for serving a personal music collection, along with a web client, a headless local client that exposes a web interface, and a command-line program for managing the data.

The basic idea is that you mirror your music collection (and the corresponding album artwork) to Google Cloud Storage and then run the nup update command against the local copy to save metadata to a Datastore database. User-generated information like ratings, tags, and playback history is also saved in Datastore. The App Engine app performs queries against Datastore and serves songs and album art from Cloud Storage.

An Android client is also available.

This project is probably only of potential interest to people who both buy all of their music and are comfortable setting up a Google Cloud project and compiling and running command-line programs, which seems like a very small set. If it sounds appealing to you and you'd like to see more detailed instructions, though, please let me know!

History

In 2001 or 2002, I wrote dmc, a silly C application that used the FMOD library to play my MP3 collection. It used OpenGL to render a UI and some simple visualizations. I ran it on a small (Mini-ITX? I don't remember) computer plugged into my TV.

Sometime around 2005 or 2006, I decided that I wanted to be able to rate and tag the songs in my music collection and track my playback history so I could listen to stuff that I liked but hadn't heard recently, or play non-distracting instrumental music while reading or programming. I was using MPD to play locally-stored MP3 files at the time, so I wrote some Ruby scripts to search for and enqueue songs and display information about the current song onscreen. I also wrote a Ruby audioscrobbler library for sending playback reports to the service that later became Last.fm.

In 2010, I decided that it was silly to need to have my desktop computer turned on whenever I wanted to listen to music, so I wrote a daemon in Ruby to serve music and album art and support searching/tagging/rating/etc. over HTTP. Song information was stored in a SQLite database. I added a web interface and wrote an Android client that supported offline playback, and ran the server on a little always-on SoC Linux device. This was before the Raspberry Pi was released, and all I remember about the device was that upgrades were terrifying because it didn't put out enough power to be able to reliably boot off its external HDD.

In 2014, I decided that it'd be nice to be less dependent on my home network connection, so I rewrote the server in Go as a Google App Engine app that'd serve music and covers from Google Cloud Storage. That's what this repository contains.

It's 2021 now and I haven't felt the urge to rewrite all this code again.

The name "nup" doesn't mean anything; it just didn't seem to be used by any major projects. (I tried to think of a backronym for it but didn't come up with anything obvious other than the 'p' standing for "player".)

Dependencies

In addition to linking against the dependencies listed in go.mod, this repository contains code, fonts, and icons from several other projects:

• cmd/nup-headless/audio/beep: beep resampling code
• cmd/nup-headless/audio/minimp3: minimp3 MP3 decoder
• web and web/fonts: MFG Labs icon set, Fontelico collection, and Roboto font
• web/icons: Google Material icons and svg-spinners:ring-resize icon by Utkarsh Verma

Please see the LICENSE files in these directories for additional information.

Configuration

At the very least, you'll need to do the following:

• Create a Google Cloud project.
• Enable the Cloud Storage and App Engine APIs.
• Create Cloud Storage buckets for your songs and album art.
• Use the gsutil tool to sync songs and album art to Cloud Storage.
• Compile the nup tool using go install ./cmd/nup.
• Deploy the App Engine app.
• Write config files for the nup tool and the app.
• Use nup update to send song metadata to the App Engine app so it can be saved to Datastore.

As mentioned above, please let me know if you're feeling adventurous and would like to see detailed instructions for these steps.

nup tool

Create a YAML file at $HOME/.nup/config.yaml corresponding to the Config struct in cmd/nup/client/config.go:

serverUrl:https://my-project-id.appspot.comusername:toolspassword:my-tools-passwordcoverDir:/home/me/music/.coversmusicDir:/home/me/musiccomputeGain:true

Server

Create a YAML file corresponding to the Config struct in server/config/config.go:

users:- email:example@gmail.com- email:example.2@gmail.com- username:androidpassword:my-android-password- username:toolspassword:my-tools-passwordadmin:truesongBucket:my-songscoverBucket:my-covers

Run nup config -set /path/to/server_config.yaml to save the server configuration to Datastore.

Deploying

To deploy the App Engine app (as defined in app.yaml) and delete old, non-serving versions, run the build/deploy.sh script.

Note that App Engine often continues serving stale versions of static files for 10 minutes or more after deploying. I think that this has been broken for a long time. This Stack Overflow question has more discussion.

After changes to index.yaml, run build/deploy.sh -i to create new Datastore indexes and delete old ones.

You should also run build/deploy.sh cron.yaml once to deploy the daily stats-updating cron job described in cron.yaml.

Development and testing

The example/ directory contains code for starting a local App Engine server with example data for development.

All tests can be executed by running go test ./... from the root of the repository.

• Unit tests live alongside the code that they exercise.
• End-to-end tests that exercise the App Engine server and the nup executable are in the test/e2e/ directory.
• Selenium tests that exercise both the web interface (in Chrome) and the server are in the test/web/ directory. By default, Chrome runs headlessly.