Repyportability in RepyV2 should allow us to import Repy files from pythonpath

[choksi81]

Currently we are not able to use dy_import_module_symbols() to import repy modules into python files even if the modules are in the pythonpath. In RepyV1, repyhelper was able to translate and import all repy files into a python file as long as the repy file was in the pythonpath. It seems we lost this feature in Repy V2 when using repyportability + dylink. This feature would be very nice for projects such as AFFIX.

[choksi81]

Author: monzum
I have attached an initial patch (repyportability_not_valid.diff) for the changes I wanted to propose. However this patch only works if the repy files are directly imported. It does not seem to work if a python file is importing a repy file, which itself imports another repy file. Below is an example of a scenario where this patch does not work.

test.py:
  # Import repyportability and initialize dylink.
  from repyportability import *
  _context = locals()
  add_dy_support(_context)

  dy_import_module_symbols("hello.repy")



hello.repy:
  dy_import_module_symbols("world.repy")



world.repy:
  log("Hello World")

Running the above code will allow us to locate hello.repy from pythonpath and attempt to import it. However the import will fail when hello.repy tries to import world.repy

User traceback:
  "/home/monzum/exdisk/work/seattle/hello", line 3, in <module>

Exception (with type 'exceptions.Exception'): Failed to locate the module! Module: 'world'

I will keep digging into this and try to figure out why this is the case.

[choksi81]

Author: monzum
Here is the full traceback of the error:

monzum@TestUbuntu:~/exdisk/work$ python test_dylink.py 
Traceback (most recent call last):
  File "test_dylink.py", line 5, in <module>
    dy_import_module_symbols("hello.repy")
  File "/home/monzum/exdisk/work/seattle/repyportability.py", line 257, in _dy_import_module_symbols
    new_context = _context[new_callfunc)._context
  File "/home/monzum/exdisk/work/seattle/repyportability.py", line 248, in _new_dy_import_module_symbols
    return original_import_module(filenamewithpath, callfunc)
  File "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 468, in _dy_import_module
    return dylink_import_module(module,import_context,new_callfunc,additional_globals)
  File "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 385, in dylink_import_module
    dylink_import_global(module_name, new_context, new_callfunc)
  File "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 341, in dylink_import_global
    raise Exception, "Caught exception while initializing module ("+module+")! Debug String: "+debug_str
Exception: Caught exception while initializing module (/home/monzum/exdisk/work/seattle/hello)! Debug String: ---
Following is a full traceback, and a user traceback.
The user traceback excludes non-user modules. The most recent call is displayed last.

Full debugging traceback:
  "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 338, in dylink_import_global
  "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 535, in evaluate
  "/home/monzum/exdisk/work/seattle/virtual_namespace.py", line 117, in evaluate
  "/home/monzum/exdisk/work/seattle/safe.py", line 522, in safe_run
  "/home/monzum/exdisk/work/seattle/hello", line 3, in <module>
  "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 471, in _dy_import_global
  "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 326, in dylink_import_global
  "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 234, in _dy_module_code

User traceback:
  "/home/monzum/exdisk/work/seattle/hello", line 3, in <module>

Exception (with type 'exceptions.Exception'): Failed to locate the module! Module: 'world'
---
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 64, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python2.7/dist-packages/apport/__init__.py", line 1, in <module>
    from apport.report import Report
  File "/usr/lib/python2.7/dist-packages/apport/report.py", line 18, in <module>
    import problem_report
  File "/usr/lib/python2.7/dist-packages/problem_report.py", line 14, in <module>
    import zlib, base64, time, sys, gzip, struct, os
  File "/usr/lib/python2.7/gzip.py", line 10, in <module>
    import io
  File "/usr/lib/python2.7/io.py", line 90, in <module>
    RawIOBase.register(FileIO)
  File "/usr/lib/python2.7/abc.py", line 107, in register
    if not isinstance(subclass, (type, types.ClassType)):
TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and types

Original exception was:
Traceback (most recent call last):
  File "test_dylink.py", line 5, in <module>
    dy_import_module_symbols("hello.repy")
  File "/home/monzum/exdisk/work/seattle/repyportability.py", line 257, in _dy_import_module_symbols
    new_context = _context['dy_import_module']('dy_import_module'](module,)(module, new_callfunc)._context
  File "/home/monzum/exdisk/work/seattle/repyportability.py", line 248, in _new_dy_import_module_symbols
    return original_import_module(filenamewithpath, callfunc)
  File "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 468, in _dy_import_module
    return dylink_import_module(module,import_context,new_callfunc,additional_globals)
  File "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 385, in dylink_import_module
    dylink_import_global(module_name, new_context, new_callfunc)
  File "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 341, in dylink_import_global
    raise Exception, "Caught exception while initializing module ("+module+")! Debug String: "+debug_str
Exception: Caught exception while initializing module (/home/monzum/exdisk/work/seattle/hello)! Debug String: ---
Following is a full traceback, and a user traceback.
The user traceback excludes non-user modules. The most recent call is displayed last.

Full debugging traceback:
  "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 338, in dylink_import_global
  "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 535, in evaluate
  "/home/monzum/exdisk/work/seattle/virtual_namespace.py", line 117, in evaluate
  "/home/monzum/exdisk/work/seattle/safe.py", line 522, in safe_run
  "/home/monzum/exdisk/work/seattle/hello", line 3, in <module>
  "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 471, in _dy_import_global
  "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 326, in dylink_import_global
  "/home/monzum/exdisk/work/seattle/dylink_repy.py", line 234, in _dy_module_code

User traceback:
  "/home/monzum/exdisk/work/seattle/hello", line 3, in <module>

Exception (with type 'exceptions.Exception'): Failed to locate the module! Module: 'world'
---

[choksi81]

Attachments:
https://github.com/SeattleTestbed/attic/blob/master/TICKET_ATTACHMENTS/repyportability_not_valid.diff