-
Notifications
You must be signed in to change notification settings - Fork 59
Sentence segmentation accuracy #157
I am using the latest build of Catalyst for sentence segmentation. I have used it on the attached file but the accuracy seems to be off. For example, the following was broken into 2 separate sentences (see General Instruction A.2. below):
June 7, 2021 MACQUARIE INFRASTRUCTURE CORPORATION (Exact name of Registrant as specified in its charter) (212) 231-1000 (Registrant's telephone number, including area code) N.A. (Former name or former address, if changes since last report) Check the appropriate box below if the Form 8-k filing is intended to simultaneously satisfy the filing obligation of the registrant under any of the following provisions (see General Instruction A.2.
below): ̈ Written communications pursuant to Rule 425 under the Securities Act (17 CFR 230.425) x Soliciting material pursuant to Rule 14a-12 under the Exchange Act (17 CFR 240.14a-12) ̈ Pre-commencement communications pursuant to Rule 14d-2(b) under the Exchange Act (17 CFR 240.14d-2(b)) ̈ Pre-commencement communications pursuant to Rule 13e-4(c) under the Exchange Act (17 CFR 240.13e-4(c)) Securities registered pursuant to Section 12(b) of the Act: Indicate by check mark whether the registrant is an emerging growth company as defined in Rule 405 of the Securities Act of 1933 (-230.405 of this chapter) or Rule 12b-2 of the Securities Exchange Act of 1934 (-240.12b-2 of this chapter).
I am using the code:
string text = File.ReadAllText("mic.txt");
Storage.Current = new OnlineRepositoryStorage(new DiskStorage("catalyst-models"));
var nlp = await Pipeline.ForAsync(Language.English);
var doc = new Document(text, Language.English);
nlp.ProcessSingle(doc);
// write the sentences to a file
using (StreamWriter sw = new StreamWriter("catalyst.sentences.txt"))
{
foreach (Span span in doc.Spans)
{
sw.WriteLine(span.Value);
sw.WriteLine();
}
}
Is there any way to improve the accuracy of this sentence segmentation?