DSL4j library

SWUbanner

Lingvo DSL is an one of popular dictionary formats. DSL4j is a parser library of Lingvo DSL dictionary file for Java 11 and later.

DSL4j supports .dsl file and also .dsl.dz compressed file. DSL4j loads all dictionary data and parse its index into memory. An index will be saved in index file specified in second argument of DslDictionary#loadDictionary method. When a specified index file exists and validated as up-to-date, dsl4j loads the index instead of parsing dictionary.

NOTICE: A current version does not support media archive file ...dsl.files.zip that is supported by GoldenDict.

dictionary format

Lingvo DSL format specification is a little ambiguous, and there are many variations of data in the wild. Here is a table to show what variations are supported.

Unsupported DSL syntax

There are several syntaxes that DSL4j cannot handle. These syntax characters are passed as-is.

• Unsorted part of the head word with braces
• Tilda word replacement
• Alternative heading section with (...)
• Subentry @ mark
• Reference to another card entry
• Comments with double braces "{{comment}}" in article

Development status

A status of library development is considered as Beta.

Install

Apache Maven

<dependency>
 <groupId>tokyo.northside</groupId>
 <artifactId>dsl4j</artifactId>
 <version>1.0.0</version>
</dependency>

Gradle Groovy DSL

implementation 'tokyo.northside:dsl4j:1.0.0'

Gradle kotlin DSL

implementation("tokyo.northside:dsl4j:1.0.0")

Scala SBT

libraryDependencies += "tokyo.northside" % "dsl4j" % "1.0.0"

Configuration

A Java Platform Module System module name is tokyo.northside.dsl4j

Usage

DSL4j provides a DSL dictionary loader and an article parser. You should call DslDictionary#loadData method to load DSL file. The method return DslDictionary object that has methods lookup and lookupPredictive. The former method search word, and the latter is predictive, run prefix search for word. These method returns DslResult object.

You need to prepare DslVisitor filter class. DSL4j provides three standard visitor filters.

• HtmlDslVisitor: convert to HTML.
• PlainDslVisitor: convert to plain text which strip all tags.
• DumpDslVisitor: produce the same content as input DSL (for debug).

DslResult.getEntries(visitor) returns List<Map.Entry<String, T>> where T is decided by visitor.

Example

Here is a simple example of usage.

publicclass Main{publicstaticvoidmain(String...argv){PathdictionaryPath=Path.to(argv[1]);PathindexPath=Path.to(dictionaryPath+".idx");Stringword=argv[2];DslDictionarydslDictionary=DslDictionary.loadDictionary(dictionaryPath,indexPath);PlainDslVisitorplainDslVisitor=newPlainDslVisitor();DslResultdslResult=dslDictionary.lookup(word);for(Map.Entry<String,String>entry:dslResult.getEntries(plainDslVisitor)){Stringkey=entry.getKey();Stringarticle=entry.getValue();System.out.println(key,article);}}}

Here is a pragmatic one with java8 streams, predictive search method, and HTML converter;

publicclass Example{publicvoidexample(){DslDictionarydslDictionary=DslDictionary.loadDictionary(file);HtmlDslVisitorhtmlDslVisitor=newHtmlDslVisitor(file.getParent());List<String>result=dslDictionary.lookupPredictive(word).getEntries(htmlDslVisitor).stream().map(e->"<p><strong>"+e.getKey()+"</strong>"+e.getValue()+"</p>").collect(Collectors.toList());}}

Please check DslDictionaryTest cases for visitor differences.

index file

The index file format is DSL4j original, but it is designed to be usable from other applications/libraries. The index file is formatted and produced by Google's Protocol Buffers and compressed by gzip.

Definition of the index file is placed in src/main/proto/DslIndex.proto with ProtoBuf v3 source format. Java sources are automatically generated at build time in build/generated/main/java

WARNING: The index file feature is in a status of experimental. It will be changed to break compatibility without a precaution notice.

Language names and codes

DSL4j has immutable tables of language names and codes DSL supported. You can get ISO639 language code from LanguageName and LanguageCode class.

with DSL4j version 1.1.0 and later

publicclass Example{voidlanguageTest(){assert(LanguageCode.getLanguageCode(1).equals("en"));assert(LangaugeName.containsLanguage("Russian"));}}

with DSL4j version 1.0.0

publicclass Example{voidlanguageTest(){LanguageCodelanguageCode=newLanguageCode();LanguageNamelanguageName=newLanguageName();assert(languageCode.get(1).equals("en"));assert(langaugeName.containsKey("Russian"));}}

These codes are in [lang id=?] tag. When you want to write your own custom visitor, you may want to know it.

Colors

DSL4j recognize HTML color names in lower case for [c] tag.

Media tags

DSL4j does not handle media tags for image, sound and video specially. DSL4j just handle [s] and [video] as a normal tag, and pass file name as ordinal text.

Please see HtmlDslVisitor class to know how to handle these tags.

DSL article syntax definition

DSL article parser is written using JavaCC parser generator and the definition is placed at src/main/java/io/github/eb4j/dsl/DslParser.jj Java sources are automatically generated at build time in build/generated/main/java

Use cases

• OmegaT: A free and open source multiplatform Computer Assisted Translation tool
• EBViewer: Basic dictionary search application

License and copyright

DSL4J is distributed under GNU General Public License version 3 or (at your option) any later version. Please see the LICENSE file for details.

A part of the code is delivered from OmegaT.

Copyright (C) 2015-2016 Hiroshi Miura, Aaron Madlon-Kay

Copyright (C) 2021-2025 Hiroshi Miura