0

I have a very big and important project but I have a problem about choosing a way to handle data.

  • database : If I use database I can sort them and filter them. Do you think that in this way filtering data is gonna be faster or I should code it in python? requesting data or filtering or any other process to database in the same time by more than one software is gonna make any problem?
  • file : I'm currently using text files to handle data and I extract data from them but I have so many threads and multiprocessing functions that should open the same file. So sometimes an error comes up and says "Error : Permission denied" which is reasonable.

So my software needs to do a lot of data processing and filtering (using some conditions) and sometimes opening and accessing data in the same time by so many functions or other python scripts, that in files it comes up with an error (Permission denied).

So how can I filter and process data without having any error. what tools should I use? I need a fast method to do all these.

Note : Each 10 seconds , data will be updated

asked Sep 3, 2018 at 17:33

1 Answer 1

1

If frequent data operation is going to be performed it is advisable to use a database over a file system

answered Sep 3, 2018 at 18:00
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

Draft saved
Draft discarded

Sign up or log in

Sign up using Google
Sign up using Email and Password

Post as a guest

Required, but never shown

Post as a guest

Required, but never shown

By clicking "Post Your Answer", you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.