You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -6,30 +6,82 @@ Download this extension from the [Visual Studio Market Place ( vs 2019 )](https:
6
6
or download from [releases](https://github.com/FortuneN/FineCodeCoverage/releases). Older versions can be obtained from [here](https://ci.appveyor.com/project/FortuneN/finecodecoverage/history).
7
7
8
8
---
9
-
Prerequisites
9
+
## Prerequisites
10
10
11
11
Only that the test adapters are nuget packages. For instance, the NUnit Test Adapter extension is not sufficient.
12
-
FCC will copy your test dll and dependencies to a sub folder this may affect your tests. The alternative is to set the option AdjacentBuildOutput to true.
12
+
13
13
14
14
---
15
15
16
-
Introduction
16
+
## Introduction
17
+
18
+
Fine Code Coverage provides code coverage using one of 3 different coverage tools. In previous releases there were two coverage tools being utilised, OpenCover and Coverlet that will be referred to as 'old coverage'.
19
+
Microsoft now provides a free coverage solution that you can choose to use by setting the Visual Studio Fine Code Coverage enumeration option RunMsCodeCoverage. This will probably be the preferred coverage
20
+
tool for most developers. It is currently in Beta.
21
+
22
+
With the old coverage it was possible for FCC to provide an abstraction over each tool's exclusion / inclusion options. This abstraction does not work for MS code coverage.
23
+
Thus you will find that there are separate configuration options for Ms coverage vs old coverage and options that are common to the two.
24
+
Configuration is available with Visual Studio options and project msbuild properties. All visual studio settings can be overridden from test project settings and some settings
25
+
can only be set in project files.
17
26
18
-
Fine Code Coverage works by reacting to the visual studio test explorer, providing coverage from each test project containing tests that you have selected
19
-
to run. This coverage is presented as a single unified report in the Fine Code Coverage Tool Window as well as coloured margins alongside your code.
20
-
This coverage is not dynamic and represents the coverage obtained from the last time you executed tests.
21
-
When the coverage becomes outdated, you can click the 'FCC Clear UI' button in Tools or run coverage again.
27
+
Regardless of the coverage tool employed the process begins with FCC reacting to the test explorer in visual studio. One of the 3 coverage tools provides the coverage results that are presented as a single unified report in the Fine Code Coverage Tool Window. The report shows line and branch coverage and risk hotspots with the facility to open your class files that will have coloured margins to indicate uncovered or partially covered code.
28
+
This coverage is not dynamic and represents the coverage obtained from the last time you executed tests. When the coverage becomes outdated, you can click the 'FCC Clear UI' button in Tools or run coverage again.
22
29
23
30
Details of how FCC is progressing with code coverage can be found in the Coverage Log tab in the Fine Code Coverage Tool Window with more detailed logs in the FCC Output Window Pane. If you experience issues then providing the logs from the output window will help to understand the nature of the problem.
24
31
32
+
## Why use MS Code Coverage ?
33
+
34
+
With the old coverage FCC needed to copy your test dll and dependencies and run OpenCover or Coverlet on those files. This is not necessary with ms code coverage.
35
+
The old coverage would wait until tests have finished before starting the coverage tool to re-run all tests. This is not necessary with ms code coverage.
36
+
The old coverage was based upon every test. Ms code coverage is coverage from the tests you select in the test explorer.
37
+
38
+
## How to utilize MS Code Coverage with FCC ?
39
+
40
+
Firstly you need to change the RunMsCodeCoverage option from No.
41
+
42
+
Ms code coverage requires a [runsettings](https://docs.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2022) file that is configured appropriately for
43
+
code coverage. This requires that you have the ms code coverage package and have pointed to it with the TestAdaptersPaths element as well as specifying the ms data collector. [Exclusions and inclusions](https://docs.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2022#include-or-exclude-assemblies-and-members)
44
+
are also specified in the runsettings. I don't think that the documentation is clear enough on how this works so you may want to look at [this issue](https://github.com/microsoft/vstest/issues/3462).
45
+
46
+
FCC does not require you to do this. If you do not provide a runsettings and RunMsCodeCoverage is Yes then FCC will generate one. If RunMsCodeCoverage is IfInRunSettings then if the project has runsettings that includes the ms data collector element configured correctly then FCC will process the collected results.
47
+
48
+
## Run settings generation from template
49
+
50
+
FCC includes the ms code coverage package and will create the necessary runsettings file for each test project being run from the test explorer window.
51
+
The exclusions and inclusions will come from visual studio options or from the project file in a similar manner to the old coverage. As ms code coverage uses regex and has different methods of exclusion / inclusion to
52
+
Coverlet and OpenCover there are ms specific Visual Studio options and project elements.
53
+
54
+
As FCC provides a runsettings file for each test project ( if you have not provided a solution wide or project specific ) it has to write the RunSettingsFilePath element in the project file.
55
+
Although FCC removes this element from the project file it will still show as git modified.
56
+
57
+
FCC creates the runsettings from a template using string replacement. If so desired you can provide your own templates. FCC will look for fcc-ms-runsettings-template.xml in the project directory or the solution directory.
58
+
Your template needs to be a valid xml document but does not need to supply all of the run settings elements. FCC will add the replaceable ResultsDirectory and TestAdaptersPaths ( and the container RunConfiguration element if necessary)
59
+
elements if not present. FCC will also add the ms DataCollector / Configuration / CodeCoverage replaceable elements if not present. It will also add the non exclusion / inclusion recommended CodeCoverage elements AllowLowIntegrityProcesses, CollectFromChildProcesses, CollectAspDotNet and UseVerifiableInstrumentation if not provided. UseVerifiableInstrumentation will be false for .Net Framework.
60
+
To see the generated run settings use the RunSettingsFilePath element in the project file.
61
+
62
+
For a custom template, that provides its own elements that would otherwise have been provided by FCC, to participate in template replacement you need to add strings of the type %fcc_replace%.
63
+
Excludes and includes follow the format "%fcc_modulepaths_exclude%". For FCC to supply the fcc provided ms code coverage path to TestAdaptersPaths use %fcc_testadapter%.
64
+
65
+
## Non template run settings interception
66
+
It is also possible to use your own runsettings file and have FCC add to it and replace. FCC will ensure that the TestAdaptersPaths element is present as well as the ms data collector. Unlike with a template, if the ms data collector is present then the xml only has replacement applied to it.
67
+
68
+
## Run settings defaults and merging
69
+
70
+
Ms code coverage does provide a default Configuration / CodeCoverage element if not provided. It will also add some default exclusions if not present or merge them in unless you add the attribute mergeDefaults='false'.
71
+
For instance it Attributes exclude ExcludeFromCodeCoverageAttribute.
72
+
If you are interested see ...\AppData\Local\FineCodeCoverage\msCodeCoverage\_version_\build\netstandard1.0\Microsoft.VisualStudio.TraceDataCollector.dll and the DynamicCoverageDataCollector.
73
+
74
+
75
+
## Problems with ms code coverage
76
+
Please check [troubleshooting](https://docs.microsoft.com/en-us/visualstudio/test/troubleshooting-code-coverage?view=vs-2022) before reporting an issue.
77
+
78
+
## Old style coverage
79
+
80
+
Dlls are copied to a sub folder of project output folder which may affect your tests. The alternative is to set the option AdjacentBuildOutput to true.
25
81
The coverage is provided by either [OpenCover](https://github.com/OpenCover/opencover) for old style projects and [Coverlet](https://github.com/coverlet-coverage/coverlet)
26
-
for new style sdk projects. FCC provides an abstraction over both so that it is possible to ignore the differences between the two but there are circumstances where
82
+
for new style sdk projects. Although FCC provides an abstraction over both so that it is possible to ignore the differences between the two but there are circumstances where
27
83
it is important to be aware of cover tool that will be run. This is most apparent when Coverlet is used, please read on for the specifics.
28
-
The other scenario would be when you want to use a specific version of the coverage tool. FCC will keep up to date with Coverlet and OpenCover
29
-
but there may be a preview version that you want to use. This can be configured.
30
-
31
-
Configuration is available with Visual Studio settings and project msbuild properties. All visual studio settings can be overridden from test project settings and some settings
32
-
can only be set in project files.
84
+
The other scenario would be when you want to use a specific version of the coverage tool. This can be configured.
33
85
34
86
The coverage tools that FCC leverages are by default installed into the FineCodeCoverage directory within `Environment.SpecialFolder.LocalApplicationData`.
35
87
This can be changed with the ToolsDirectory Visual Studio option. Ensure that this containing directory exists and upon restart the tools will be installed within.
@@ -82,6 +134,9 @@ Run a(some) unit test(s) and ...
82
134
<IncludeTestAssembly>
83
135
True
84
136
</IncludeTestAssembly>
137
+
<ModulePathsExclude>
138
+
.*Fabrikam.Math.UnitTest.dll
139
+
</ModulePathsExclude>
85
140
</PropertyGroup>
86
141
```
87
142
@@ -117,54 +172,39 @@ If you are using option 1) then project and global options will only be used whe
117
172
118
173
#### Options
119
174
```
175
+
*** Common
120
176
CoverageColoursFromFontsAndColours Specify true to use Environment / Fonts and Colors / Text Editor for editor Coverage colouring ( if present).
121
177
Coverage Touched Area / Coverage Not Touched Area / Coverage Partially Touched Area.
122
178
When false colours used are Green, Red and Gold.
123
-
Enabled Specifies whether or not coverage output is enabled
124
-
RunInParallel By default tests run and then coverage is performed. Set to true to run coverage immediately
125
-
RunWhenTestsFail By default coverage runs when tests fail. Set to false to prevent this. **Cannot be used in conjunction with RunInParallel**
126
-
RunWhenTestsExceed Specify a value to only run coverage based upon the number of executing tests. **Cannot be used in conjunction with RunInParallel**
127
-
Exclude Filter expressions to exclude specific modules and types (multiple values)
128
-
Include Filter expressions to include specific modules and types (multiple values)
129
-
IncludeReferencedProjects Set to true to add all referenced projects to Include.
130
-
ExcludeByFile Glob patterns specifying source files to exclude e.g. **/Migrations/* (multiple values)
131
-
ExcludeByAttribute Attributes to exclude from code coverage (multiple values)
132
-
IncludeTestAssembly Specifies whether to report code coverage of the test assembly
179
+
180
+
FCCSolutionOutputDirectoryName To have fcc output visible in a sub folder of your solution provide this name
181
+
182
+
ToolsDirectory Folder to which copy tools subfolder. Must alredy exist. Requires restart of VS.
133
183
134
184
ThresholdForCyclomaticComplexity When [cyclomatic complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity) exceeds this value for a method then the method will be present in the risk hotspots tab.
135
-
ThresholdForNPathComplexity When [npath complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity) exceeds this value for a method then the method will be present in the risk hotspots tab. OpenCover only.
136
-
ThresholdForCrapScore When [crap score](https://testing.googleblog.com/2011/02/this-code-is-crap.html) exceeds this value for a method then the method will be present in the risk hotspots tab. OpenCover only.
137
185
138
186
StickyCoverageTable Set to true for coverage table to have a sticky thead.
139
187
NamespacedClasses Set to false to show classes in report in short form. Affects grouping.
140
188
HideFullyCovered Set to true to hide classes, namespaces and assemblies that are fully covered.
141
189
142
-
RunSettingsOnly Specify false for global and project options to be used for coverlet data collector configuration elements when not specified in runsettings
143
-
CoverletCollectorDirectoryPath Specify path to directory containing coverlet collector files if you need functionality that the FCC version does not provide.
190
+
Enabled Specifies whether or not coverage output is enabled
191
+
RunWhenTestsFail By default coverage runs when tests fail. Set to false to prevent this. **Cannot be used in conjunction with RunInParallel**
192
+
RunWhenTestsExceed Specify a value to only run coverage based upon the number of executing tests. **Cannot be used in conjunction with RunInParallel**
193
+
RunMsCodeCoverage Change to IfInRunSettings to only collect with configured runsettings. Yes for runsettings generation.
144
194
145
-
CoverletConsoleLocal Specify true to use your own dotnet tools local install of coverlet console.
146
-
CoverletConsoleCustomPath Specify path to coverlet console exe if you need functionality that the FCC version does not provide.
147
-
CoverletConsoleGlobal Specify true to use your own dotnet tools global install of coverlet console.
195
+
IncludeTestAssembly Specifies whether to report code coverage of the test assembly
196
+
IncludeReferencedProjects Set to true to add all referenced projects to Include.
148
197
149
-
FCCSolutionOutputDirectoryName To have fcc output visible in a sub folder of your solution provide this name
198
+
*** OpenCover / Coverlet
150
199
AdjacentBuildOutput If your tests are dependent upon their path set this to true.
151
200
152
-
ToolsDirectory Folder to which copy tools subfolder. Must alredy exist. Requires restart of VS.
153
-
154
-
The "CoverletConsole" settings have precedence Local / CustomPath / Global.
155
-
156
-
Both 'Exclude' and 'Include' options can be used together but 'Exclude' takes precedence.
157
-
158
-
You can ignore a method or an entire class from code coverage by creating and applying the [ExcludeFromCodeCoverage] attribute present in the System.Diagnostics.CodeAnalysis namespace.
159
-
You can also ignore additional attributes by adding to the 'ExcludeByAttributes' list (short name or full name supported) e.g. :
160
-
[GeneratedCode] => Present in System.CodeDom.Compiler namespace
161
-
[MyCustomExcludeFromCodeCoverage] => Any custom attribute that you may define
162
-
163
-
164
-
```
201
+
Exclude Filter expressions to exclude specific modules and types (multiple values)
202
+
Include Filter expressions to include specific modules and types (multiple values)
203
+
ExcludeByFile Glob patterns specifying source files to exclude e.g. **/Migrations/* (multiple values)
204
+
ExcludeByAttribute Attributes to exclude from code coverage (multiple values)
205
+
RunInParallel By default OpenCover / Coverlet tests run and then coverage is performed. Set to true to run coverage immediately
165
206
166
-
#### Filter Expressions
167
-
```
207
+
Filter expressions
168
208
Wildcards
169
209
* => matches zero or more characters
170
210
@@ -174,7 +214,44 @@ Examples
174
214
[*]Coverlet.Core.Instrumentation.* => All types belonging to Coverlet.Core.Instrumentation namespace in any assembly
175
215
[coverlet.*.tests]* => All types in any assembly starting with coverlet. and ending with .tests
176
216
217
+
177
218
Both 'Exclude' and 'Include' options can be used together but 'Exclude' takes precedence.
219
+
220
+
You can ignore a method or an entire class from code coverage by creating and applying the [ExcludeFromCodeCoverage] attribute present in the System.Diagnostics.CodeAnalysis namespace.
221
+
You can also ignore additional attributes by adding to the 'ExcludeByAttributes' list (short name or full name supported) e.g. :
222
+
[GeneratedCode] => Present in System.CodeDom.Compiler namespace
223
+
[MyCustomExcludeFromCodeCoverage] => Any custom attribute that you may define
224
+
225
+
*** MS Code Coverage each multiple regexes to be transformed into runsettings elements
226
+
ModulePathsExclude
227
+
ModulePathsInclude
228
+
CompanyNamesExclude
229
+
CompanyNamesInclude
230
+
PublicKeyTokensExclude
231
+
PublicKeyTokensInclude
232
+
SourcesExclude
233
+
SourcesInclude
234
+
AttributesExclude
235
+
AttributesInclude
236
+
FunctionsExclude
237
+
FunctionsInclude
238
+
239
+
*** Coverlet
240
+
RunSettingsOnly Specify false for global and project options to be used for coverlet data collector configuration elements when not specified in runsettings
241
+
CoverletCollectorDirectoryPath Specify path to directory containing coverlet collector files if you need functionality that the FCC version does not provide.
242
+
CoverletConsoleLocal Specify true to use your own dotnet tools local install of coverlet console.
243
+
CoverletConsoleCustomPath Specify path to coverlet console exe if you need functionality that the FCC version does not provide.
244
+
CoverletConsoleGlobal Specify true to use your own dotnet tools global install of coverlet console.
245
+
246
+
The "CoverletConsole" settings have precedence Local / CustomPath / Global.
247
+
248
+
*** OpenCover
249
+
OpenCoverCustomPath Specify path to open cover exe if you need functionality that the FCC version does not provide.
250
+
ThresholdForNPathComplexity When [npath complexity](https://en.wikipedia.org/wiki/Cyclomatic_complexity) exceeds this value for a method then the method will be present in the risk hotspots tab. OpenCover only.
251
+
ThresholdForCrapScore When [crap score](https://testing.googleblog.com/2011/02/this-code-is-crap.html) exceeds this value for a method then the method will be present in the risk hotspots tab. OpenCover only.
0 commit comments