Update Selenium, RF, Firefox and friends

[hvelarde]

The goal of this pull request it to update all components related with browser testing.

It's not clear how to achieve that on a simple step.

refs. plone/plone.app.robotframework#71

[hvelarde]

@idgserpro @claytonc @flipmcf @datakurre @fredvd can you help here?

[datakurre]

At least https://github.com/collective/collective.ifttt has working Travis-CI -configuration.

…

On 15 Jun 2018, 16.23 +0300, Héctor Velarde ***@***.***>, wrote: @idgserpro @claytonc @flipmcf @datakurre @fredvd can you help here? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[hvelarde]

just a quick update I was able to run the tests on this branch locally like this:

$ wget https://github.com/mozilla/geckodriver/releases/download/v0.20.1/geckodriver-v0.20.1-linux64.tar.gz
...
$ firefox -v
Mozilla Firefox 61.0
$ PATH=~/collective/cover/:$PATH bin/test --layer=Robot
Running collective.cover.testing.collective.cover:Robot tests:
  Set up plone.testing.zca.LayerCleanup in 0.000 seconds.
  Set up plone.testing.z2.Startup in 0.241 seconds.
  Set up plone.app.testing.layers.PloneFixtureCould not install product plone.autoform
 in 6.222 seconds.
  Set up collective.cover.testing.Fixture in 4.380 seconds.
  Set up plone.app.robotframework.remote.AutoLoginRemoteLibrary:RobotRemote in 0.005 seconds.
  Set up plone.testing.z2.ZServer in 0.502 seconds.
  Set up collective.cover.testing.collective.cover:Robot in 0.000 seconds.
  Running:
    5/23 (21.7%)[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
    10/23 (43.5%)
==============================================================================
Test Layout                                                                   
==============================================================================
Test Basic Layout Operations                                          | FAIL |
ElementClickInterceptedException: Message: Element <div class="cssclasswidget-overlay"> is not clickable at point (640,438) because another element <li> obscures it
------------------------------------------------------------------------------
Test Layout                                                           | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================
Output:  /home/hvelarde/collective/cover/parts/test/test_layout/Test_Basic_Layout_Operations/output.xml



Failure in test Test Basic Layout Operations (test_layout.robot)
Traceback (most recent call last):
  File "/home/hvelarde/collective/buildout.python/parts/opt/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/home/hvelarde/.buildout/eggs/robotsuite-2.0.0-py2.7.egg/robotsuite/__init__.py", line 458, in runTest
    assert last_status == 'PASS', last_message
AssertionError: ElementClickInterceptedException: Message: Element <div class="cssclasswidget-overlay"> is not clickable at point (640,438) because another element <li> obscures it


    12/23 (52.2%)[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
    17/23 (73.9%)
==============================================================================
Test List Tile                                                                
==============================================================================
Test List Tile                                                        | FAIL |
Test news item != My document
------------------------------------------------------------------------------
Test List Tile                                                        | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================
Output:  /home/hvelarde/collective/cover/parts/test/test_list_tile/Test_List_Tile/output.xml



Failure in test Test List Tile (test_list_tile.robot)
Traceback (most recent call last):
  File "/home/hvelarde/collective/buildout.python/parts/opt/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/home/hvelarde/.buildout/eggs/robotsuite-2.0.0-py2.7.egg/robotsuite/__init__.py", line 458, in runTest
    assert last_status == 'PASS', last_message
AssertionError: Test news item != My document

    21/23 (91.3%)[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
[ WARN ] Using 'Get Element Attribute' without explicit attribute is deprecated.
                  
  Ran 23 tests with 2 failures and 0 errors in 12 minutes 5.890 seconds.
Tearing down left over layers:
  Tear down collective.cover.testing.collective.cover:Robot in 0.000 seconds.
  Tear down plone.app.robotframework.remote.AutoLoginRemoteLibrary:RobotRemote in 0.005 seconds.
  Tear down plone.testing.z2.ZServer in 4.587 seconds.
  Tear down collective.cover.testing.Fixture in 0.017 seconds.
  Tear down plone.app.testing.layers.PloneFixture in 0.077 seconds.
  Tear down plone.testing.z2.Startup in 0.005 seconds.
  Tear down plone.testing.zca.LayerCleanup in 0.004 seconds.

[hvelarde]

@datakurre seems it was working since the beginning; I'm skipping the tests that failed locally and now the build is passing in Travis.

now we need to define a canonical way to download and install geckodriver.

[idgserpro]

now we need to define a canonical way to download and install geckodriver.

A "manual" way is available in https://community.plone.org/t/is-anyone-here-using-newer-versions-of-firefox-48-and-selenium-for-robot-tests-whats-your-configuration-to-make-it-work-and-avoid-selenium-common-exceptions-webdriverexception-message-cant-load-the-profile/6263/4. The disadvantage is that you may need to change it from time to time to keep with the latest in firefox. Maybe a script that always gets the latest driver from https://github.com/mozilla/geckodriver/releases would suffice.

I suggest creating a bash script that does this and upload it to buildout.plonetest, so every package in the Plone world would benefit from it.

[hvelarde]

perfect! we need to fix the failing test before merging this.

[hvelarde]

@idgserpro @claytonc there are some RF tests failing with this configuration but I don't understand why; if you can take a look I will appreciate.

[idgserpro]

I don't understand either... a local testing is needed to see why Page should have contained element 'xpath: //input[contains(text(), "Log in")]' but did not. is happening. Maybe if changed to Wait until page contains instead of Page should contain?

[idgserpro]

@hvelarde I think these versions are only compatible with Plone 5. See this commit:

plone/plone.app.robotframework@160275e

that is in the same PR.

[idgserpro]

Try update only the Selenium in Plone 4.3.

[hvelarde]

Capture Page Screenshot keyword has been replaced by Take Screenshot in robotframework 3.0; I was not able to make it work on my Ubuntu machine because some OS missing dependencies.

I would appreciate if somebody else have time to help me fixing the remaining failing tests.