API compatibility check

[chcg]

See notepad-plus-plus/notepad-plus-plus#5741.

Maybe consider to add a CI check for API compatilibity of the plugins from the list with new N++ versions to at least avoid crashes of N++ with existing plugins on startup.

[donho]

That's a good idea.
But how?

[chcg]

What I currently have in mind is to start n++ with the plugin to test and check that the startup doesn't result in a coredump/crash by a appveyor testconfig. No interactive test with the plugin menu. So that would be just a basic test for the compatibility of the plugins.
Maybe the plugin system of N++ could be extended to introduce an API version in https://github.com/notepad-plus-plus/notepad-plus-plus/blob/master/PowerEditor/src/MISC/PluginsManager/PluginInterface.h which could be requested from the plugin.
Plugins already have the possibility to request the N++ version via NPPM_GETNPPVERSION. So maybe that is already sufficient for plugin developers to adapt behaviour to a specific N++ version range if necessary.

[donho]

@chcg

What I currently have in mind is to start n++ with the plugin to test and check that the startup doesn't result in a coredump/crash by a appveyor testconfig. No interactive test with the plugin menu. So that would be just a basic test for the compatibility of the plugins.

Yes, good idea. You can copy Notepad++ last official release on the fly for it.

Maybe the plugin system of N++ could be extended to introduce an API version in https://github.com/notepad-plus-plus/notepad-plus-plus/blob/master/PowerEditor/src/MISC/PluginsManager/PluginInterface.h which could be requested from the plugin.

I will see what I can do about it.

[chcg]

@ArkadiuszMichalski Could we go on with that at #250. Do you have a script available which could be used for the appveyor CI job?

[ArkadiuszMichalski]

@chcg
Do you have a script available which could be used for the appveyor CI job?

Probably yes, because I'm using NodeJS. I want it to work locally as well, so that everyone can test for themselves, or use it to download only all plugins (.zip) and, for example, check with antivirus.

NodeJS is more portable than Python (especially on Win7), but you can port this script to another language if necessary. This is just a preliminary version, to see how to write it at all, and try it locally.

When I'm done (I have to add some configuration options) I'll create a new repo for downloads all files + some initiall instructions. You will be able to test and adapt the whole thing under appveyor CI job. I will have some more questions regarding crash detection, but I will ask them after created this repo.

[chcg]

Check with antivirus is also an open issue #53.

As download of the plugins is already happening by
https://github.com/notepad-plus-plus/nppPluginList/blob/master/validator.py#L126
and also the unzipping
https://github.com/notepad-plus-plus/nppPluginList/blob/master/validator.py#L143

I would hope from there it would be just a small step to place it into a N++ plugin folder and test execution.

[ArkadiuszMichalski]

@chck Nothing fancy, but it works. Now it's just only for experiments.
https://github.com/ArkadiuszMichalski/NPP_tools
You can test locally in a safe way, just provide some short list of "safe plugins" (eg. which you manage yourself).

I have a few questions about closing NPP, because some plugins generate a popup, which requires a response. So:

• If there is a crash, it is easy to spot (no popup).
• If the popup will appear, how to detect it and close NPP? I can close the process with some delay and detect that it's not a crash, but it can't be done any better?

What am I doing now? I run NPP with -multiInst and -export=functionList arguments and if there is no crash and no popup, NPP closes by itself. If there is a popup, I try to close it (but I use an additional program for this - NirCMD), and after they are closed, the NPP also closes itself. Thanks to this, the test runs very quickly. However, I would prefer not to use an additional program. We can probably call some native Windows commands in NodeJS to close popup, but that requires some extra searching how to do it. Or, as I wrote above, simply close the NPP process with some permanent delay.

And one more, when I use -export=functionList argument NPP will close after loading all .dll files from plugins? Because I don't know if this trick can be used in the context of plugin testing or not.