Re: Serialized test data in table format?
[
Date Prev][
Date Next][
Thread Prev][
Thread Next]
[
Date Index]
[
Thread Index]
- Subject: Re: Serialized test data in table format?
- From: Sean Conner <sean@...>
- Date: 2017年9月28日 17:51:07 -0400
It was thus said that the Great Russell Haley once stated:
> Hi,
>
> I'm starting some testing on my lua-persist library that serializes
> tables into lmdb. I suck at creating test data and keep getting
> stalled out on the task. I thought I'd ask if anyone has some
> serialized tables they could offer for my testing?
>
> One problem so far is that this thing is blazing fast. I had one set
> of data that was around 6 MB and the system didn't even blink. If
> anyone can offer 50-100MB of data (or more!) I'd be REALLY grateful.
At one time, I had an 83MB Lua script that generated a single file [1] but
alas, that is long gone. I just now generated a 44MB Lua script by crawling
over some directories on my computer [2]:
local dump = require "org.conman.table".dump
local fsys = require "org.conman.fsys"
local start = arg[1] or os.getenv("HOME")
local function generate(path)
local data = {}
local folders = {}
local oldcwd,err = fsys.getcwd()
fsys.chdir(path)
local dir = fsys.opendir()
while true do
local entry = dir:next()
if not entry then break end
local info = fsys.stat(entry)
if info then
if info.type == 'file' then
data[entry] = info
elseif info.type == 'dir' then
table.insert(folders,entry)
end
end
end
dir:__gc() -- erm ... yea, kinda need this
for _,name in ipairs(folders) do
data[name] = generate(name)
end
fsys.chdir(oldcwd)
return data
end
local x = generate(start)
dump("x",x)
If you don't feel like install my modules [3] you can probably get
something similar with lposix or luafilesystem or something.
-spc
[1] http://lua-users.org/lists/lua-l/2009-11/msg00654.html
[2] Found a resource leak (well, kind of, the GC doesn't kick in as
often as it should) dealing with reading directories. Ah, large
data ... how well you find bugs!
[3] https://github.com/spc476/lua-conmanorg