Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

Commit cc85bd0

Browse files
author
Jay Conrod
committed
testing, cmd/go: clarify documentation
This CL removes 'go help fuzz' but expands the testing package documentation with much of the same information. It also removes documentation for the unimplemented -keepfuzzing flag and makes a number of other clarifications. Addressing comments in CL 348469. Updates #46629 Change-Id: I12ab5971c900c2e43f2c2d83c6705e8cd642388b Reviewed-on: https://go-review.googlesource.com/c/go/+/351113 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Trust: Katie Hockman <katie@golang.org> Trust: Bryan C. Mills <bcmills@google.com>
1 parent fa5c504 commit cc85bd0

File tree

5 files changed

+98
-89
lines changed

5 files changed

+98
-89
lines changed

‎src/cmd/go/alldocs.go‎

Lines changed: 21 additions & 35 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎src/cmd/go/internal/clean/clean.go‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,9 @@ The -modcache flag causes clean to remove the entire module
7575
download cache, including unpacked source code of versioned
7676
dependencies.
7777
78-
The -fuzzcache flag causes clean to remove values used for fuzz testing.
78+
The -fuzzcache flag causes clean to remove files stored in the Go build
79+
cache for fuzz testing. Files stored in source testdata directories
80+
are left in place.
7981
8082
For more about build flags, see 'go help build'.
8183

‎src/cmd/go/internal/test/test.go‎

Lines changed: 18 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -210,10 +210,10 @@ control the execution of any test:
210210
(for example, -benchtime 100x).
211211
212212
-count n
213-
Run each test, benchmark, and fuzz targets' seed corpora n times
214-
(default 1).
213+
Run each test, benchmark, and fuzz seed n times (default 1).
215214
If -cpu is set, run n times for each GOMAXPROCS value.
216-
Examples are always run once.
215+
Examples are always run once. -count does not apply to
216+
fuzz targets matched by -fuzz.
217217
218218
-cover
219219
Enable coverage analysis.
@@ -242,14 +242,18 @@ control the execution of any test:
242242
-cpu 1,2,4
243243
Specify a list of GOMAXPROCS values for which the tests, benchmarks or
244244
fuzz targets should be executed. The default is the current value
245-
of GOMAXPROCS.
245+
of GOMAXPROCS. -cpu does not apply to fuzz targets matched by -fuzz.
246246
247247
-failfast
248248
Do not start new tests after the first test failure.
249249
250-
-fuzz name
251-
Run the fuzz target with the given regexp. Must match exactly one fuzz
252-
target. This is an experimental feature.
250+
-fuzz regexp
251+
Run the fuzz target matching the regular expression. When specified,
252+
the command line argument must match exactly one package, and regexp
253+
must match exactly one fuzz target within that package. After tests,
254+
benchmarks, seed corpora of other fuzz targets, and examples have
255+
completed, the matching target will be fuzzed. See the Fuzzing section
256+
of the testing package documentation for details.
253257
254258
-fuzztime t
255259
Run enough iterations of the fuzz test to take t, specified as a
@@ -262,9 +266,6 @@ control the execution of any test:
262266
Log verbose output and test results in JSON. This presents the
263267
same information as the -v flag in a machine-readable format.
264268
265-
-keepfuzzing
266-
Keep running the fuzz target if a crasher is found.
267-
268269
-list regexp
269270
List tests, benchmarks, fuzz targets, or examples matching the regular
270271
expression. No tests, benchmarks, fuzz targets, or examples will be run.
@@ -275,10 +276,13 @@ control the execution of any test:
275276
Allow parallel execution of test functions that call t.Parallel, and
276277
f.Fuzz functions that call t.Parallel when running the seed corpus.
277278
The value of this flag is the maximum number of tests to run
278-
simultaneously. While fuzzing, the value of this flag is the
279-
maximum number of workers to run the fuzz function simultaneously,
280-
regardless of whether t.Parallel has been called; by default, it is set
281-
to the value of GOMAXPROCS.
279+
simultaneously.
280+
While fuzzing, the value of this flag is the maximum number of
281+
subprocesses that may call the fuzz function simultaneously, regardless of
282+
whether T.Parallel is called.
283+
By default, -parallel is set to the value of GOMAXPROCS.
284+
Setting -parallel to values higher than GOMAXPROCS may cause degraded
285+
performance due to CPU contention, especially when fuzzing.
282286
Note that -parallel only applies within a single test binary.
283287
The 'go test' command may run tests for different packages
284288
in parallel as well, according to the setting of the -p flag
@@ -507,28 +511,6 @@ See the documentation of the testing package for more information.
507511
`,
508512
}
509513

510-
var HelpFuzz = &base.Command{
511-
UsageLine: "fuzz",
512-
Short: "fuzzing",
513-
Long: `
514-
By default, go test will build and run the fuzz targets using the target's seed
515-
corpus only. Any generated corpora in $GOCACHE that were previously written by
516-
the fuzzing engine will not be run by default.
517-
518-
When -fuzz is set, the binary will be instrumented for coverage. After all
519-
tests, examples, benchmark functions, and the seed corpora for all fuzz targets
520-
have been run, go test will begin to fuzz the specified fuzz target.
521-
Note that this feature is experimental.
522-
523-
-run can be used for testing a single seed corpus entry for a fuzz target. The
524-
regular expression value of -run can be in the form $target/$name, where $target
525-
is the name of the fuzz target, and $name is the name of the file (ignoring file
526-
extensions) to run. For example, -run=FuzzFoo/497b6f87.
527-
528-
See https://golang.org/s/draft-fuzzing-design for more details.
529-
`,
530-
}
531-
532514
var (
533515
testBench string // -bench flag
534516
testC bool // -c flag

‎src/cmd/go/main.go‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,6 @@ func init() {
8080
modfetch.HelpPrivate,
8181
test.HelpTestflag,
8282
test.HelpTestfunc,
83-
test.HelpFuzz,
8483
modget.HelpVCS,
8584
}
8685
}

‎src/testing/testing.go‎

Lines changed: 56 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -134,28 +134,68 @@
134134
//
135135
// Fuzzing
136136
//
137-
// Functions of the form
137+
// 'go test' and the testing package support fuzzing, a testing technique where
138+
// a function is called with randomly generated inputs to find bugs not
139+
// anticipated by unit tests.
140+
//
141+
// A fuzz target is a function that declares a set of "seed" inputs by calling
142+
// F.Add, then provides a fuzz function by calling F.Fuzz. A fuzz target has
143+
// the form:
144+
//
138145
// func FuzzXxx(*testing.F)
139-
// are considered fuzz targets, and are executed by the "go test" command. When
140-
// the -fuzz flag is provided, the functions will be fuzzed.
141146
//
142-
// For a description of the testing flags, see
143-
// https://golang.org/cmd/go/#hdr-Testing_flags.
147+
// For example:
148+
//
149+
// func FuzzHex(f *testing.F) {
150+
// for _, seed := range [][]byte{{}, {0}, {9}, {0xa}, {0xf}, {1, 2, 3, 4}} {
151+
// f.Add(seed)
152+
// }
153+
// f.Fuzz(func(t *testing.T, in []byte) {
154+
// enc := hex.EncodeToString(in)
155+
// out, err := hex.DecodeString(enc)
156+
// if err != nil {
157+
// t.Fatalf("%v: decode: %v", in, err)
158+
// }
159+
// if !bytes.Equal(in, out) {
160+
// t.Fatalf("%v: not equal after round trip: %v", in, out)
161+
// }
162+
// })
163+
// }
164+
//
165+
// Seed inputs may be registered by calling F.Add or by storing files in the
166+
// directory testdata/fuzz/<Name> (where <Name> is the name of the fuzz target)
167+
// within the package containing the fuzz target. Seed inputs are optional, but
168+
// the fuzzing engine may find bugs more efficiently when provided with a set
169+
// of small seed inputs with good code coverage.
170+
//
171+
// The fuzz function provided to F.Fuzz must accept a *testing.T parameter,
172+
// followed by one or more parameters for random inputs. The types of arguments
173+
// passed to F.Add must be identical to the types of these parameters. The fuzz
174+
// function may signal that it's found a problem the same way tests do: by
175+
// calling T.Fail (or any method that calls it like T.Error or T.Fatal) or by
176+
// panicking.
177+
//
178+
// When fuzzing is enabled (by setting the -fuzz flag to a regular expression
179+
// that matches a specific fuzz target), the fuzz function is called with
180+
// arguments generated by repeatedly making random changes to the seed inputs.
181+
// On supported platforms, 'go test' compiles the test executable with fuzzing
182+
// coverage instrumentation. The fuzzing engine uses that instrumentation to
183+
// find and cache inputs that expand coverage, increasing the liklihood of
184+
// finding bugs. If the fuzz function finds a problem, the fuzzing engine writes
185+
// the inputs that caused the problem to a file in the directory
186+
// testdata/fuzz/<Name> within the package directory. This file later serves as
187+
// a seed input. If the file can't be written at that location (for example,
188+
// because the directory is read-only), the fuzzing engine writes the file to
189+
// the fuzz cache directory within the build cache instead.
190+
//
191+
// When fuzzing is disabled, the fuzz function is called with the seed inputs
192+
// registered with F.Add and seed inputs from testdata/fuzz/<Name>. In this
193+
// mode, the fuzz target acts much like a regular test, with subtests started
194+
// with F.Fuzz instead of T.Run.
144195
//
145-
// For a description of fuzzing, see golang.org/s/draft-fuzzing-design.
146196
// TODO(#48255): write and link to documentation that will be helpful to users
147197
// who are unfamiliar with fuzzing.
148198
//
149-
// A sample fuzz target looks like this:
150-
//
151-
// func FuzzBytesCmp(f *testing.F) {
152-
// f.Fuzz(func(t *testing.T, a, b []byte) {
153-
// if bytes.HasPrefix(a, b) && !bytes.Contains(a, b) {
154-
// t.Error("HasPrefix is true, but Contains is false")
155-
// }
156-
// })
157-
// }
158-
//
159199
// Skipping
160200
//
161201
// Tests or benchmarks may be skipped at run time with a call to

0 commit comments

Comments
(0)

AltStyle によって変換されたページ (->オリジナル) /