[openstack-dev] [MagnetoDB] Best practices for uploading large amounts of data
Maksym Iarmak
miarmak at mirantis.com
Fri Mar 28 10:39:16 UTC 2014
Hi guys,
I suggest taking a look, how Swift and Ceph do such things.
2014年03月28日 12:33 GMT+02:00 Maksym Iarmak <miarmak at mirantis.com>:
>>>> 2014年03月28日 11:29 GMT+02:00 Serge Kovaleff <skovaleff at mirantis.com>:
>> Hi Iliia,
>>>> I would take a look into BSON http://bsonspec.org/
>>>> Cheers,
>> Serge Kovaleff
>>>> On Thu, Mar 27, 2014 at 8:23 PM, Illia Khudoshyn <ikhudoshyn at mirantis.com
>> > wrote:
>>>>> Hi, Openstackers,
>>>>>> I'm currently working on adding bulk data load functionality to
>>> MagnetoDB. This functionality implies inserting huge amounts of data
>>> (billions of rows, gigabytes of data). The data being uploaded is a set of
>>> JSON's (for now). The question I'm interested in is a way of data
>>> transportation. For now I do streaming HTTP POST request from the client
>>> side with gevent.pywsgi on the server side.
>>>>>> Could anybody suggest any (better?) approach for the transportation,
>>> please?
>>> What are best practices for that.
>>>>>> Thanks in advance.
>>>>>> --
>>>>>> Best regards,
>>>>>> Illia Khudoshyn,
>>> Software Engineer, Mirantis, Inc.
>>>>>>>>>>>> 38, Lenina ave. Kharkov, Ukraine
>>>>>> www.mirantis.com <http://www.mirantis.ru/>
>>>>>> www.mirantis.ru
>>>>>>>>>>>> Skype: gluke_work
>>>>>> ikhudoshyn at mirantis.com
>>>>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140328/2fdfafdb/attachment.html>
More information about the OpenStack-dev
mailing list