Quicksql: QSQL是以SQL进行单一、混合查询的一款产品

【升级通知】Gitee 将于 7 月 11 日晚 23 点进行全站升级维护

代码拉取完成,页面将自动刷新

richgiteeai

Watch

不关注关注所有动态仅关注版本发行动态关注但不提醒动态

1 Star 0 Fork 40

xianghaizing/Quicksql

forked from Gitee 极速下载/Quicksql

代码 Wiki 统计流水线

服务

加入 Gitee

与超过 1400万开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)

免费加入

已有帐号? 立即登录

克隆/下载

HTTPS SSH SVN SVN+SSH 下载ZIP

提示

下载代码请复制以下命令到终端执行

为确保你提交的代码身份被 Gitee 正确识别,请执行以下命令完成配置

git config --global user.name userName 
git config --global user.email userEmail

初次使用 SSH 协议进行代码克隆、推送等操作时,需按下述提示完成 SSH 配置

1 生成 RSA 密钥

2 获取 RSA 公钥内容,并配置到 SSH公钥中

在 Gitee 上使用 SVN,请访问使用指南

使用 HTTPS 协议时,命令行会出现如下账号密码验证步骤。基于安全考虑,Gitee 建议配置并使用私人令牌替代登录密码进行克隆、推送等操作

Username for 'https://gitee.com': userName

Password for 'https://userName@gitee.com': # 私人令牌

新建文件

新建子模块

上传文件

分支 41

标签 5

贡献代码

同步代码

创建 Pull Request

了解更多

对比差异通过 Pull Request 同步

同步更新到分支

通过 Pull Request 同步

将会在向当前分支创建一个 Pull
Request,合入后将完成同步

dependabot[bot] Bump log4j-api from 2.16.0 to 2.17.0 4418afc

199 次提交

.github

assembly

bin

client

conf

core

data/sales

docker

docs

example

meta

metastore

parser

server

.gitignore

LICENSE

NOTICE

README.md

mkdocs.yml

pom.xml

README

MIT

[English](./README_doc.md)|[中文](./docs/zh/index.md)

![200_200](docs/images/logo.jpeg)

![Language](https://img.shields.io/github/languages/top/qihoo360/quicksql?style=flat)
![Release Version](https://img.shields.io/github/v/release/Qihoo360/quicksql)
![license](https://img.shields.io/badge/license-MIT-blue.svg?style=flat)
![Documentation Status](https://readthedocs.org/projects/quicksql/badge/?version=latest)
![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)

Quicksql is a SQL query product which can be used for specific datastore queries or multiple datastores correlated queries. It supports relational databases, non-relational databases and even datastore which does not support SQL (such as Elasticsearch, Druid) . In addition, a SQL query can join or union data from multiple datastores in Quicksql. For example, you can perform unified SQL query on one situation that a part of data stored on Elasticsearch, but the other part of data stored on Hive. The most important is that QSQL is not dependent on any intermediate compute engine, users only need to focus on data and unified SQL grammar to finished statistics and analysis.

[![Star-History](docs/images/star-history.png)](https://star-history.t9t.io/#Qihoo360/Quicksql)

### Architecture

An architecture diagram helps you access Quicksql more easily.

![1540973404791](docs/images/p1.png)

QSQL architecture consists of three layers:

- Parsing Layer: Used for parsing, validation, optimization of SQL statements, splitting of mixed SQL and finally generating Query Plan;

- Computing Layer: For routing query plan to a specific execution plan, then interpreted to executable code for given storage or engine(such as Elasticsearch JSON query or Hive HQL);

- Storage Layer: For data prepared extraction and storage;

### Basic Features

In the vast majority of cases, we expect to use a language for data analysis and don't want to consider things that are not related to data analysis, Quicksql is born for this.

The goal of Quicksql is to provide three functions:

***1. Unify all structured data queries into a SQL grammar***

- **Only Use SQL**

In Quicksql, you can query Elasticsearch like this:

``````sql
SELECT state, pop FROM geo_mapping WHERE state = 'CA' ORDER BY state
``````

Even an aggregation query:

``````sql
SELECT approx_count_distinct(city), state FROM geo_mapping GROUP BY state LIMIT 10
``````

You won't be annoyed again because the brackets in the JSON query can't match ;)

- **Eliminate Dialects**

In the past, the same semantic statement needs to be converted to a dialect for different engines, such as:

```sql
SELECT * FROM geo_mapping -- MySQL Dialect
LIMIT 10 OFFSET 10 
```

``````sql
SELECT * FROM geo_mapping -- Oracle Dialect
OFFSET 10 ROWS FETCH NEXT 10 ROWS ONLY 
``````

In Quicksql, relational databases no longer have the concept of dialects. You can use the grammar of Quicksql to query any engine, just like this:

```sql
SELECT * FROM geo_mapping LIMIT 10 OFFSET 10 -- Run Anywhere
```

***2. Shield the isolation between different data sources***

Consider a situation where you want to join tables that are in different engines or are not in the same cluster, you may be in trouble.

However, in Quicksql, you can query like this:

``````sql
SELECT * FROM 
 (SELECT * FROM es_raw.profile AS profile //index.tpye on Elasticsearch 
 WHERE note IS NOT NULL )AS es_profile
INNER JOIN 
 (SELECT * FROM hive_db.employee AS emp //database.table on Hive
 INNER JOIN hive_db.action AS act //database.table on Hive
 ON emp.name = act.name) AS tmp 
ON es_profile.prefer = tmp.prefer
``````

***3. Choose the most appropriate way to execute the query***

A query involving multiple engines can be executed in a variety of ways. Quicksql wants to combine the advantages of each engine to find the most appropriate one.

### Getting Started

For instructions on building Quicksql from source, see [Getting Started](./docs/reference/getting-started.md).

### Reporting Issues

If you find any bugs or have any better suggestions, please file a [GitHub issue](https://github.com/Qihoo360/Quicksql/issues).

And if the issue is approved, a label [QSQL-ID] will be added before the issue description by committer so that it can correspond to commit. Such as:

```sql
[QSQL-1002]: Views generated after splitting logical plan are redundant.
```

### Contributing

We welcome contributions.

If you are interested in Quicksql, you can download the source code from GitHub and execute the following maven command at the project root directory:

```shell
mvn -DskipTests clean package
```

If you are planning to make a large contribution, talk to us first! It helps to agree on the general approach. Log a Issures on GitHub for your proposed feature.

Fork the GitHub repository, and create a branch for your feature.

Develop your feature and test cases, and make sure that `mvn install` succeeds. (Run extra tests if your change warrants it.)

Commit your change to your branch.

If your change had multiple commits, use `git rebase -i master` to squash them into a single commit, and to bring your code up to date with the latest on the main line.

Then push your commit(s) to GitHub, and create a pull request from your branch to the QSQL master branch. Update the JIRA case to reference your pull request, and a committer will review your changes.

The pull request may need to be updated (after its submission) for two main reasons:

1. you identified a problem after the submission of the pull request;
2. the reviewer requested further changes;

In order to update the pull request, you need to commit the changes in your branch and then push the commit(s) to GitHub. You are encouraged to use regular (non-rebased) commits on top of previously existing ones.

## Join us

[![Slack](docs/images/slack.png)](https://join.slack.com/t/quicksql/shared_invite/enQtODkwMzM0Njc3NTExLWQxNjRlY2M5YTlkMTk4OTM2YzhjMjUxYTUyN2VlNzJlNzQwM2E4YjkxNzA4MDllODg5NWUxNDY4MTMyMzczMWI)
[![Github](docs/images/github.png)](https://github.com/qihoo360/Quicksql/issues)
[![QQ](docs/images/qq.png)](https://jq.qq.com/?_wv=1027&k=5782R6F)

MIT License Copyright (c) 2018 <QSql Project> Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.