Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

add ability to define the size of the database in the constructor #298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
missvalentinep wants to merge 2 commits into sql-js:master
base: master
Choose a base branch
Loading
from gemini-testing:html-reporter

Conversation

Copy link

@missvalentinep missvalentinep commented Oct 3, 2019

When we have to work with big data, the ability to predefine the size of a new database can significantly speed up script execution time, because multiple calls to MEMF.expandFileStorage is way less efficient than one "bigger" call to it.

JavaScript profile of merging 7 databases (each of size ~15 MB) into one, without expanding the final db file first
Screen Shot 2019年10月03日 at 10 51 31 AM

The same operation but with expanding the final database to the size of (sum of 7 databases' sizes)
Screen Shot 2019年10月03日 at 10 52 01 AM

miripiruni reacted with thumbs up emoji brody4hire reacted with eyes emoji
@missvalentinep missvalentinep changed the title (削除) add ability to define the size of the database the constructor (削除ここまで) (追記) add ability to define the size of the database in the constructor (追記ここまで) Oct 3, 2019
Copy link

@lovasoa could you pay attention to this change please? This is not just optimization. initialDbSize allows us to merge several tables in a acceptable amount of time. The union is too slow without it.

Thank you!

Copy link
Member

lovasoa commented Oct 11, 2019
edited
Loading

The structure of this repo makes it very difficult to review changes. I am currently not merging pull requests by lack of time.
I cannot just merge this kind of PR because they contain a lot of minified JS that could contain malicious code.

One thing to do that would make it much easier to make changes is to handle compilation to JS by a continuous integration tool, and maybe use a more popular language such as JS of TypeScript.

Copy link
Author

What if I removed the compiled files from pull request and you compiled them yourself or left the compilation to prepublish command?

brody4hire reacted with thumbs up emoji

Copy link

@lovasoa OK, I totally agree. Internet is dangerous place.

To be sure there are no malware in minified JS: May I ask you to take diff from src/api.coffee only and make dist/* by yourself?

Copy link

miripiruni commented Oct 11, 2019
edited
Loading

I cannot just merge this kind of PR because they contain a lot of minified JS that could contain malicious code.

The fact is we also are tremble because it's hard to understand that dist/* from latest version currently does not contains any suspicious code.

Copy link
Member

lovasoa commented Oct 11, 2019

What would both reduce the maintenance burden (which is the biggest problem here) and improve security is having a continuous integration service bundling the files.

Copy link
Member

lovasoa commented Oct 13, 2019

@kripken : maybe you can activate GitHub actions for this repo? https://github.com/features/actions

Copy link
Collaborator

kripken commented Oct 13, 2019

@lovasoa all I see is a way to request being added to the beta - is there something else? I did that now.

Copy link
Member

lovasoa commented Oct 14, 2019

No, nothing else. Once you are added to the beta, we can add a yaml file in .github/workflows with the scripts we want to run.

Copy link
Collaborator

kripken commented Oct 15, 2019

I got an email that I am in the beta, so maybe it can work now...

Copy link
Member

lovasoa commented Oct 16, 2019
edited
Loading

Thank you Alon. I created a github workflow that compiles the code, tests it, and uploads the resulting artifacts: https://github.com/kripken/sql.js/actions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

AltStyle によって変換されたページ (->オリジナル) /