SoftClient4ES Logo

🏗️ Architecture

⚡ Quick Start — REPL Client

Installation

curl -fsSL https://raw.githubusercontent.com/SOFTNETWORK-APP/SoftClient4ES/main/scripts/install.sh | bash

irm https://raw.githubusercontent.com/SOFTNETWORK-APP/SoftClient4ES/main/scripts/install.ps1 | iex

Connect and Query

softclient4es --host localhost --port 9200

-- Create a table (index)
CREATE TABLE users (
 id KEYWORD,
 name TEXT FIELDS(
 raw KEYWORD
 ) OPTIONS (fielddata = true),
 email KEYWORD,
 age INTEGER,
 created_at DATE,
 PRIMARY KEY (id)
);
-- Insert data
INSERT INTO users (id, name, email, age) 
VALUES ('1', 'Alice', 'alice@example.com', 30);
-- Query with SQL
SELECT name, email, age FROM users WHERE age > 25 ORDER BY name;
-- Update records
UPDATE users SET age = 31 WHERE id = '1';
-- Show tables
SHOW TABLES LIKE 'user%';

🎯 Why SoftClient4ES?

📋 SQL Support

DDL — Data Definition Language

CREATE TABLE products (
 id KEYWORD,
 name TEXT FIELDS(
 raw KEYWORD
 ) OPTIONS (fielddata = true),
 email KEYWORD,
 price DOUBLE,
 tags KEYWORD,
 PRIMARY KEY (id)
);
ALTER TABLE products ADD COLUMN stock INTEGER;
DESCRIBE TABLE products;
DROP TABLE old_products;
TRUNCATE TABLE logs;

DML — Data Manipulation Language

INSERT INTO products (id, name, price) VALUES ('p1', 'Laptop', 999.99);
UPDATE products SET price = 899.99 WHERE id = 'p1';
DELETE FROM products WHERE price < 10;
COPY INTO products FROM '/data/products.json';

DQL — Data Query Language

SELECT name, price, COUNT(*) as sales
FROM products
WHERE category = 'electronics'
GROUP BY name, price
HAVING COUNT(*) > 10
ORDER BY sales DESC
LIMIT 100;

Materialized Views

CREATE OR REPLACE MATERIALIZED VIEW orders_with_customers_mv
REFRESH EVERY 10 SECONDS
WITH (delay = '2s', user_latency = '1s')
AS
SELECT
 o.id,
 o.amount,
 c.name AS customer_name,
 c.email,
 UPPER(c.name) AS customer_name_upper
FROM orders AS o
JOIN customers AS c ON o.customer_id = c.id
WHERE o.status = 'completed';
-- Query like a regular table
SELECT * FROM orders_with_customers_mv WHERE customer_name = 'Alice';
-- Inspect
DESCRIBE MATERIALIZED VIEW orders_with_customers_mv;
SHOW CREATE MATERIALIZED VIEW orders_with_customers_mv;
SHOW MATERIALIZED VIEW STATUS orders_with_customers_mv;

🔌 JDBC Driver

Driver Setup

JDBC URL: jdbc:elastic://localhost:9200
Driver class: app.softnetwork.elastic.jdbc.ElasticDriver

Maven / Gradle / sbt

<dependency>
 <groupId>app.softnetwork.elastic</groupId>
 <artifactId>softclient4es8-jdbc-driver</artifactId>
 <version>0.1.4</version>
</dependency>

implementation 'app.softnetwork.elastic:softclient4es8-jdbc-driver:0.1.4'

libraryDependencies += "app.softnetwork.elastic" % "softclient4es8-jdbc-driver" % "0.1.4"

🏹 Arrow Flight SQL

Server Setup (Docker)

docker run -p 32010:32010 \
 -e ES_HOST=elasticsearch \
 -e ES_PORT=9200 \
 -e ES_USER=elastic \
 -e ES_PASSWORD=changeme \
 softnetwork/softclient4es8-arrow-flight-sql:latest

Python + DuckDB

import adbc_driver_flightsql.dbapi as flight_sql
import duckdb
conn = flight_sql.connect("grpc://localhost:32010")
cursor = conn.cursor()
cursor.execute("SELECT * FROM ecommerce")
table = cursor.fetch_arrow_table() # zero-copy Arrow table
duckdb.sql("SELECT category, SUM(total_price) AS revenue FROM table GROUP BY category")

Live Demo

DuckDB + Python pipeline

docker compose --profile duckdb up

Apache Superset BI dashboards

docker compose --profile superset-flight up

Grafana dashboards

docker compose --profile grafana up

🛠️ Scala Library Integration

Client Library Matrix

sbt Setup

// build.sbt
resolvers += "Softnetwork" at "https://softnetwork.jfrog.io/artifactory/releases/"
// Choose your Elasticsearch version
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es8-java-client" % "0.19.0"
// Add the community extensions for materialized views (optional)
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-community-extensions" % "0.1.4"
// Add the JDBC driver if you want to use it from Scala (optional)
libraryDependencies += "app.softnetwork.elastic" %% "softclient4es-jdbc-driver" % "0.1.4"

import app.softnetwork.elastic.client._
val client = ElasticClientFactory.create()
// SQL queries
val results = client.search(SQLQuery("SELECT * FROM users WHERE age > 25"))
// Type-safe queries with compile-time validation
case class User(id: String, name: String, age: Int)
val users: Source[User, NotUsed] = client.scrollAs[User](
 "SELECT id, name, age FROM users WHERE active = true"
)

✨ Key Features

🔀 Zero-Downtime Mapping Migration

client.updateMapping("users", newMapping) // Handles backup, reindex, and rollback

📦 High-Performance Bulk Operations

client.bulkFromFile("/data/products.parquet", format = Parquet, idKey = Some("id"))

🔍 Smart Scroll API

client.scroll(SQLQuery("SELECT * FROM logs WHERE level = 'ERROR'"))
 .runWith(Sink.foreach(processDocument))

🔗 Akka Persistence Integration

📚 Documentation

📦 Editions and Licensing

Feature Matrix

Elasticsearch License Requirements

🗺️ Roadmap

📝 Blog Posts

🤝 Contributing

📄 License

💬 Support