Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Adding flatfile.csv.reader implementation and tests; fix a bug in flatfile.fixedlength where column match goes wrong. #178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jf-tech merged 2 commits into master from csv2r
Sep 18, 2022

Conversation

@jf-tech
Copy link
Owner

@jf-tech jf-tech commented Sep 18, 2022

The bug is in the flatfile.fixedlength.reader.linesToNode:

func (r *reader) linesToNode(decl *EnvelopeDecl, n int) *idr.Node {
	if len(r.linesBuf) < n {
		panic(
			fmt.Sprintf("linesBuf has %d lines but requested %d lines to convert",
				len(r.linesBuf), n))
	}
	node := idr.CreateNode(idr.ElementNode, decl.Name)
	for col := range decl.Columns {
		colDecl := decl.Columns[col]
		for i := 0; i < n; i++ {
			if !colDecl.lineMatch(i, r.linesBuf[i].b) {
				continue
			}
			colNode := idr.CreateNode(idr.ElementNode, colDecl.Name)
			idr.AddChild(node, colNode)
			colVal := idr.CreateNode(idr.TextNode, colDecl.lineToColumnValue(r.linesBuf[i].b))
			idr.AddChild(colNode, colVal)
			break <==== PREVIOUSLY MISSED.
		}
	}
	return node
}

v1.0.3 release missed the crucial break statement, thus for any multiple-line envelope (whether its rows based or header/footer based), if there are multiple lines to match a column, always the last one wins. This is not what our past expectation/specification was: the first line that matches should win. It's fixed now and tests have been amended to catch the issue.

...flatfile.fixedlength` where column match goes wrong.
The bug is in the `flatfile.fixedlength.reader.linesToNode`:
```
func (r *reader) linesToNode(decl *EnvelopeDecl, n int) *idr.Node {
	if len(r.linesBuf) < n {
		panic(
			fmt.Sprintf("linesBuf has %d lines but requested %d lines to convert",
				len(r.linesBuf), n))
	}
	node := idr.CreateNode(idr.ElementNode, decl.Name)
	for col := range decl.Columns {
		colDecl := decl.Columns[col]
		for i := 0; i < n; i++ {
			if !colDecl.lineMatch(i, r.linesBuf[i].b) {
				continue
			}
			colNode := idr.CreateNode(idr.ElementNode, colDecl.Name)
			idr.AddChild(node, colNode)
			colVal := idr.CreateNode(idr.TextNode, colDecl.lineToColumnValue(r.linesBuf[i].b))
			idr.AddChild(colNode, colVal)
			break <==== PREVIOUSLY MISSED.
		}
	}
	return node
}
```
v1.0.3 release missed the crucial `break` statement, thus for any multiple-line envelope (whether its rows based
or header/footer based), if there are multiple lines to match a column, always the last one wins. This is not
what our past expectation/specification was: the first line that matches should win.
Copy link

codecov bot commented Sep 18, 2022
edited
Loading

Codecov Report

Merging #178 (9f7ec63) into master (77370e7) will not change coverage.
The diff coverage is 100.00%.

@@ Coverage Diff @@
## master #178 +/- ##
==========================================
 Coverage 100.00% 100.00% 
==========================================
 Files 51 52 +1 
 Lines 2836 2971 +135 
==========================================
+ Hits 2836 2971 +135 
Impacted Files Coverage Δ
extensions/omniv21/fileformat/flatfile/csv/decl.go 100.00% <100.00%> (ø)
...tensions/omniv21/fileformat/flatfile/csv/reader.go 100.00% <100.00%> (ø)
.../omniv21/fileformat/flatfile/fixedlength/reader.go 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@jf-tech jf-tech merged commit ba93495 into master Sep 18, 2022
@jf-tech jf-tech deleted the csv2r branch September 18, 2022 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wangjia007bond wangjia007bond Awaiting requested review from wangjia007bond

@liangxibing liangxibing Awaiting requested review from liangxibing

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

AltStyle によって変換されたページ (->オリジナル) /