Summary: How can I best write tests that override socket-based communication for a library, capturing what is sent over the socket, and simulating responses based on what is sent?
Details
I'm adding a new feature to ruby-mpd, and it's extensive enough that I'd like to add tests covering many scenarios.
The library communicates with MPD using a socket connection, pushing and receiving streams of information, parsing that information and presenting it helpfully.
Although MPD is stateful, I'm not interesting in testing the stateful nature. (That's up to the MPD team.) Instead, I want to test:
- when I invoke certain code, specific strings are sent over the socket
- when I invoke that code—and my test returns a reasonable response over the socket—that the results of parsing that response are what I expect
I do not want to have to instantiate an MPD server and mock song library to run these tests.
Example
For example, I need to test that this code...
ids = @mpd.command_list(:values) do
clear
mysongs.each{ |song| addid(song) }
end
...causes this to be sent over the socket:
command_list_begin
clear
addid foo
addid bar
addid jim
command_list_end
And then I need to simulate that after sending that it receives this over the socket...
Id: 1060
Id: 1061
Id: 1062
OK
...and that, as a result, the return value ids is [1060,1061,1062].
What I've Done
My current plan (which may not be the best one) is to replace the real socket with a delegator that intercepts puts and gets. There's a recording socket to associate streams of commands and associate them with responses. There's a playback socket that consumes this association and returns lines of responses from the previous stream of commands.
This setup does not require that sets of commands be issued in the same order, but does require that each set of commands is issued exactly the same.
Current Code
Recorder
class RecordingSocket
def initialize(real_socket)
@socket = real_socket
@current = @command = []
@responses = { @command=>[] }
end
def puts(*a)
@socket.puts(*a).tap{ @command.concat(a.empty? ? [nil] : a) }
end
def gets
@socket.gets.tap do |result|
unless @command.empty?
@current,@command=@command,[]
@responses[@current] = []
end
@responses[@current] << result
end
end
def save_recording(filename=nil)
File.open(filename || "socket_recording-#{Time.now.strftime '%Y%m%d_%H%M%S'}.marshal",'wb'){ |f| f << Marshal.dump(@responses) }
end
def method_missing(*a)
@socket.send(*a)
end
end
class RecordingMPD < MPD
def socket
@recording_socket ||= RecordingSocket.new(super)
end
def save_recording(filename=nil)
@recording_socket.save_recording(filename)
end
end
Recording Responses
m = RecordingMPD.new('music.local').tap(&:connect)
begin
songs = %w[test1.mp3 test2.mp3 test3.mp3]
m.command_list do
clear
s.each{ |f| addid(f) }
end
m.queue
m.playlists.find{ |pl| pl.name=='user-gkistner' }.songs
ensure
m.save_recording 'mpd-responses.marshal'
end
Playback Setup
require_relative '../lib/ruby-mpd'
class PlaybackSocket
def initialize(filename=nil)
@responses={}
@command = []
load_recording(filename) if filename
end
def load_recording(filename)
@responses = Marshal.load(File.open(filename,'rb',&:read))
self
end
def last_messages
@current
end
def puts(*a)
@command.concat(a.empty? ? [nil] : a)
end
def gets
@current,@command=@command,[] unless @command.empty?
if @responses[@current]
@responses[@current].shift
else
raise "PlaybackSocket has no (further) recording for #{@current}"
end
end
def method_missing(*a)
raise "PlaybackSocket has no support for #{a.shift}(#{a.map(&:inspect).join(', ')})"
end
end
class PlaybackMPD < MPD
attr_reader :socket
def initialize( socket_recording_file=nil )
super()
load_recording(socket_recording_file) if socket_recording_file
end
def load_recording(filename)
@socket = PlaybackSocket.new.load_recording(filename)
self
end
def last_messages
@socket.last_messages
end
end
Running the Tests
require 'minitest/autorun'
class TestQueue < MiniTest::Unit::TestCase
def setup
@mpd = PlaybackMPD.new 'mpd-responses.marshal'
end
def test_songs
songs = @mpd.queue
assert_equal ["playlistinfo"], @mpd.last_messages
assert_equal 5, songs.length
assert songs.all?{ |value| value.is_a? MPD::Song }
assert_equal [273, 289, 129, 258, 347], songs.map(&:track_length)
end
def test_command_lists
ids = @mpd.command_list(:values) do
clear
%w[test1.mp3 test2.mp3 test3.mp3].each{ |f| addid(f) }
end
assert_equal(
["command_list_begin", "clear", "addid test1.mp3", "addid test2.mp3",
"addid test3.mp3", "command_list_end"],
@mpd.last_messages
)
assert_equal [101,102,103], ids
end
end
Specific Questions
- Is there a better way to simulate sample data from the socket?
- Is there a better way to record the series-of-puts and associate them with the responses I get? I feel like there might be a bug lurking with that
unless @command.empty?logic.
1 Answer 1
I've found that instead of a monolithic recording hash and inscrutable .marshal file, it's far easier to have the library write each puts/gets set as a file on disk in plain text. Binary file formats are like premature optimization.
The library uses a directory to store files in. I've made the name of the file not matter (so that you can name them sensibly), and put the commands into the file itself. While I still have a recorder library, this also makes it easy to hand-edit test data files, if you know exactly the strings that should be sent.
For example, here is the result of telnetting to the server and issuing one command (note that the server spits out one response before any commands are sent):
$ telnet 0 6600
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
OK MPD 0.19.0
status # here's the command I send
volume: -1
repeat: 0
random: 0
single: 0
consume: 1
playlist: 2090
playlistlength: 0
mixrampdb: 0.000000
state: stop
OK
...and here are the test files that mock this behavior:
hello.recording
--putsabove--getsbelow--
OK MPD 0.19.0
status.recording
status
--putsabove--getsbelow--
volume: -1
repeat: 0
random: 0
single: 0
consume: 1
playlist: 2090
playlistlength: 0
mixrampdb: 0.000000
state: stop
OK
Here's the library code so far:
require 'fileutils' # used for mkdir_p
require 'digest' # used to generate unique file names
# A library for testing socket-based applications.
#
# Allows you to create a socket that records +puts+ commands
# and uses those to decide the (pre-recorded) responses to
# yield for subsequent calls to +gets+.
module SocketSpoof
# The line in each recording file separating commands and responses
SPLITTER = "--putsabove--getsbelow--\n"
# Socket wrapper that generates 'recording' files consumed
# by SocketSpoof::Player.
#
# To use, replace your own socket with a call to:
#
# @socket = SocketSpoof::Recorder.new( real_socket )
#
# This will (by default) create a directory named "socket_recordings"
# and create files within there for each sequence of +puts+ followed
# by one or more gets.
class Recorder
# @param socket [Socket] The real socket to use for communication.
# @param directory [String] The directory to store recording files in.
def initialize(socket,directory:"socket_recordings")
@socket = socket
@commands = []
FileUtils.mkdir_p( @directory=directory )
end
def puts(*a)
@socket.puts(*a).tap{ @commands.concat(a.empty? ? [nil] : a) }
end
def gets
@socket.gets.tap do |response|
unless @file && @commands.empty?
@file = File.join( @directory, Digest::SHA256.hexdigest(@commands.inspect) )
File.open(@file,'w'){ |f| f.puts(@commands); f<<SPLITTER }
@commands=[]
end
File.open(@file,'a'){ |f| f.puts response }
end
end
def method_missing(*a)
@socket.send(*a)
end
end
# Socket stand-in using files on disk to send responses.
#
# A SocketSpoot::Player uses a sequence of calls to +puts+ along
# with playback files to decide what to send back when +gets+
# is called.
#
# Simply replace your normal socket instance with a Player, and
# point that player to a directory where recording files are stored.
#
# @socket = SocketSpoof::Player.new( directory:'test_data' )
#
# The name of each recording file in the directory does not matter;
# name them as you like to make them easier to find.
# The format of the files must have zero or more lines of command
# strings, followed by the +SPLITTER+ string, followed by zero or
# more lines of response strings. For example:
#
# prepare
# listplaylists
# --putsabove--getsbelow--
# playlist: Mix Rock Alternative Electric
# Last-Modified: 2015年11月23日T15:58:51Z
# playlist: Enya-esque
# Last-Modified: 2015年11月18日T16:19:12Z
# playlist: RecentNice
# Last-Modified: 2015年12月01日T15:52:38Z
# playlist: Dancetown
# Last-Modified: 2015年11月18日T16:19:26Z
# playlist: Piano
# Last-Modified: 2015年11月18日T16:17:13Z
# OK
#
# With the above file in place in the directory:
#
# @socket = SocketSpoof::Player.new
# @socket.puts "prepare"
# @socket.puts "listplaylists"
# loop do
# case [email protected]
# when "OK\n",nil then puts "all done!"
# else puts response
# end
#
# ...will output all lines from the file. As with a normal
# socket, the call to +gets+ will include a newline at the end
# of the response.
#
# If your code calls +gets+ before it ever calls +puts+, you
# will need a file with no content above the +SPLITTER+ line.
#
# To verify that your library sent the commands that you expected,
# the +last_messages+ method returns an array of strings sent to
# +puts+ since the last call to +gets+.
class Player
# @param directory [String] the name of the directory to find recordings in; defaults to "socket_recordings".
# @param auto_update [Boolean] whether the directory should be rescanned (slow!) before each call to +gets+; defaults to +false+.
def initialize(directory:"socket_recordings",auto_update:false)
@commands = []
FileUtils.mkdir_p( @directory=directory )
@auto_update = auto_update
@response_line = -1
rescan
end
# Find out what messages were last sent to the socket.
#
# Returns an array of strings sent to +puts+ since the
# last time +gets+ was called on the socket.
# @return [Array<String>] messages previously sent through +puts+
def last_messages
@current
end
def puts(*a)
@commands.concat(a.empty? ? [nil] : a)
@response_line = -1
nil # match the return value of IO#puts, just in case
end
def gets
rescan if @auto_update
@current,@commands=@commands,[] unless @commands.empty?
if @responses[@current]
@responses[@current][@response_line+=1]
else
raise "#{self.class} has no recording for #{@current}"
end
end
def method_missing(*a)
raise "#{self.class} has no support for #{a.shift}(#{a.map(&:inspect).join(', ')})"
end
private
def rescan
@responses = {}
Dir[File.join(@directory,'*')].each do |file|
commands,responses = File.open(file,'r:utf-8',&:read).split(SPLITTER,2)
if responses
@responses[commands.split("\n")] = responses.lines.to_a
else
warn "#{self.class} ignoring #{file} because it does not appear to have #{SPLITTER.inspect}."
end
end
end
end
end