Vetting of plugins

[dail8859]

Are the plugins added to this list vetted at all?

Notepad++ makes sure the list of plugins is signed so it knows it is downloading the expected bytes, but is any if the DLL files etc actually ran through a virus checker at all or something?

[chcg]

@dail8859 I do a short local test to check that the plugins are starting up and behave "normally". Obviously I can't test the complete functionality of each plugin. So there is some portion of hoping the plugin authors are gentle and don't do evil things.

But it is not the case that I ran the download through something like https://www.virustotal.com.

Do you have any ideas for this? Any maybe also some automated tests which could be added to the appveyor run?

[dail8859]

No concrete ideas. Was just curious if there was anybody checking the plugins getting put into the list. Especially ones that do not have the source code available. I know open source is no guarantee a plugin is safe but at least one can inspect the source for any blatant security issues.

[chcg]

See e.g. https://www.virustotal.com/de/documentation/public-api/ with python examples how to use it for the plugin dlls/zips binaries.
Sourcecode inspection of the plugins would require something like https://www.owasp.org/index.php/Source_Code_Analysis_Tools, e.g. SonarQube

[ArkadiuszMichalski]

@donho @chcg Any thoughts on this? This virustotal.com looks interesting, but we have Public vs Premium API, and Public is limited:

The Public API is limited to 500 requests per day and a rate of 4 requests per minute.
The Public API must not be used in commercial products, services or business workflows.

I don't know how it looks like in Premium and how to get it (how much does it cost or gets it in some other way).

Basically how this test should look like? Every time all .zip files should be submitted for new analysis? Or, if they have already been analyzed, then just verify the report? The report for a single URL .zip file also includes the SHA-256 of that file, as we have in the plugin list.
https://www.virustotal.com/gui/url/5385c63049c027c8349c014ac478ec86d7451e2b601b8c70e797b92906085fc0/details

When adding a new plug-in or a new version, isn't it worth adding the report url to the plug-in list? It would lighten the process a bit and shift some of the task for the authors. Plugins are not too big so it shouldn't be too much trouble for their authors. Having a report ready, it is easier to analyze it, and it gives at least a bit of security.

I played a bit of analyzing all reports for x86 plugins right in the browser (using a script). Unfortunately cant't be integrate with AppVeyor due to re-captcha (I only had one, but there was). For now, it can be tested locally if someone wants, at least until something else is invented.

Result:

Plugins: 160
Safe-BAD: 3
Hash-BAD: 45

More details in npp_plugin_scan.txt file. This Safe-BAD is not so bad, because only 1 scanner from 84 (or 85) does not pass.

[62] linesort Safe-BAD 1/85 Hash-OK SHA-256 bcc6d3f6a7bc5a1578b8007b4bd40ea5721dfae8875670fff8a5ff895f3c0013 https://www.virustotal.com/gui/url/e39ace438130a96a72e41069194923423ce5ad93414219ccfb297a988fc18532/details

[145] Tidy2 Safe-BAD 1/84 Hash-OK SHA-256 59cf24719009f0a62ad4414faa9b5dac7dfd637c12a13c024492fee1a5eca3ab https://www.virustotal.com/gui/url/3ec0226c72e5287a120deef8400e68210c9b51532d2b97e4ec3293f08d164267/details

[158] XPatherizerNPP Safe-BAD 1/85 Hash-OK SHA-256 92e8393a2bf94b27a8084d8d84734616a56ea51f877bf5cc62b6d494642e76b5 https://www.virustotal.com/gui/url/f8d2f04582231920987e5e5428fd7eb067473a18e0de3fcc7913d522617a51ac/details

Edit: There is also link using SHA-256 of this .zip file. Example for 3P plugin:
https://www.virustotal.com/gui/file/cb3499305fe835975a6b3917e481b0c8f38aacd4eeee9c0782ee44ed8cd08a28/details.

This package hash is unique for each version, even the smallest change? Report from .zip file hash can be treated the same as report for .zip URL (assuming that both have same hash)?
https://www.virustotal.com/gui/url/f1c2811c72c0c02419ca41ef0ecbd19756e996ef5b57bf1c4644cbf8c8c12a24/details

If so, it would simplify my analysis because I could just use the url with .zip file hash (not try finding report for .zip url from plugin list), assuming that it has always been generated (after adding a new plug-in or changing the version). Currently, I am simply searching for a report for a given URL form the plugin list (but these are extra steps ).

If the report existed, but for example is old, it can be repeated. I will write a script that will repeat the scan for reports older than the specified time (e.g. a month). I will run it from time to time so the reports will be generated for the current signatures.

Later I will then experiment with this public API to see what these limits for requests look like in practice. Directly in the browser, it is not that restrictive, but it is because of the re-captcha (it will probably popup with many to fast requests).

[chcg]

• Python module:
  https://github.com/gawen/virustotal
• github action:
  https://github.com/marketplace/actions/virustotal-github-action

[chcg]

See gh action example:
https://github.com/blu3mania/npp-papyrus/releases
via
https://github.com/crazy-max/ghaction-virustotal

[ArkadiuszMichalski]

Yes, but this limits with public API...

There are a few things about this tool. It allows to scan urls and files. Other types of scanners are used for this actions, but both analyzes are recommended.
https://support.virustotal.com/hc/en-us/articles/115002092509-Some-URL-scanner-detects-a-given-URL-but-its-corresponding-antivirus-solution-does-not-detect-the-downloaded-file-or-vice-versa

Regarding files, it can scan .zip or individual files. Scanning the .zip itself requires less effort, but checking all its files is more accurate (to reduce the time you can skip text, photos, or other "usually safe" file types).

I'm writing about it because you can get different results for these 3 scanning variants. Sometimes it detects something in 1, but not in 2 and 3, sometimes in 2, but not in 1 and 3, etc.

Another thing is how do you want to interpret the potential detection? 1/80 or 2/80 what will it mean? Plugin if not safe?

Here is a small piece of data which I have collected. I checked the 32 and 64 bit list (test URL, .zip and all files inside .zip). Result older than a month first got a rescan. Sumarize:

npp_plugin_scan_x86.txt

Plugin List version: 1.3.0
Plugin List arch: 32
Plugins: 160

Generation date: Thu, 01 Apr 2021 09:36:34 GMT
Duration time : 66 [min] 7 [s]
Steps: 160/160 [current / total]

Safe-URL-BAD: 3
Hash-URL-BAD: 0
Safe-ZIP-BAD: 16

All files: 1467
Safe-FILE-BAD: 42
Safe-FILE-NOREPORT: 0

Plugins with some problems: 40

["3P","ActiveX","Autosave2","BookmarksDook","combine","CSScriptNpp","dbgpPlugin","NppEditorConfig","FingerText","GOnpp","HTMLTag_unicode","NPPJSONViewer","LanguageHelp","linefilter2","linesort","NavigateTo","NppPluginDemo","NotepadStarterPlugin","nppConverter","NppCalc","NppEventExec","NppExport","NppQrCode32","ccc","PoorMansTSqlFormatterNppPlugin","pork2sausage","PreviewHTML","print","rdmd-en-x86","rdmd-ja-x86","NppSaveAsAdmin","SecurePad","selectNLaunch","ShtirlitzNppPlugin","sql","TagLEET","NppTextFX","Tidy2","XPatherizerNPP","zoomdisabler"]

npp_plugin_scan_x64.txt

Plugin List version: 1.3.0
Plugin List arch: 64
Plugins: 117

Generation date: Thu, 01 Apr 2021 12:57:14 GMT
Duration time : 36 [min] 39 [s]
Steps: 117/117 [current / total]

Safe-URL-BAD: 0
Hash-URL-BAD: 0
Safe-ZIP-BAD: 3

All files: 1253
Safe-FILE-BAD: 28
Safe-FILE-NOREPORT: 0

Plugins with some problems: 21

["3P","ActiveX","BigFiles","CodeAlignmentNpp","CSScriptNpp","MusicPlayer_1.0.11x64","NppPluginDemo","NotepadStarterPlugin","nppConverter","NppEventExec","NppGist","NppQrCode64","PoorMansTSqlFormatterNppPlugin","pork2sausage","PreviewHTML","NppSaveAsAdmin","selectNLaunch","SQLinFormNpp64","NppTaskList","TopMost","zoomdisabler_x64"]

You can analyze the above files yourself. Just search for something by the label, e.g. Safe-URL-BAD or Safe-FILE-BAD. For each task you have some data and a link to the report, you can quickly navigate there and check it too.

[FILE] NetFrameworkChecker.exe SHA-256 8f4f4521f77c8e8710532aa9b35191dcaee3adfd53dbead642e856b8e1eeb879
Safe-FILE-BAD: 1/69
Size uncompressed: 88.00 KB
File path: /NetFrameworkChecker.exe
Last scan: "2021-03-29 07:38:06 UTC" "1 hour ago"
Report: https://www.virustotal.com/gui/file/8f4f4521f77c8e8710532aa9b35191dcaee3adfd53dbead642e856b8e1eeb879/details

Checking all every time (for each PR) is pointless (takes a long time and rescan takes much much longer), This should be done once before generating a new list. It would also be good to put such reports somewhere for a given version of the plugins list (when new version is released). But even these results don't make it clear whether something is actually dangerous or not. One or two detections may simply be false positive.