Python3 Open PR by arpitn30 · Pull Request #5 · arpitn30/xbmc

arpitn30 · 2017-07-07T20:36:49Z

Kodi code ported to Python3 with system dependencies updated accordingly

Description

The following changes have been made to the code -

PyString has been changed to PyUnicode and PyBytes accordingly.
PyInt has been changed to PyLong since former is no longer supported.
Obsolete Py_TPFLAGS have been removed or updated
Compare has been completely replaced in favor of RichCompare
Module Initialization has been updated
- Py_InitModule has been replaced with PyModule_Create
- PyModuleDef added
- Added PyImport_AppendInittab to add modules to the import modules list
- Updated swig/CMakeLists to add Python Binding directly to Core Library
Changed charset from char to wchar_t due to new functions using the latter
Replaced PyFile_AsFile, now removed with a custom function
Updated GIL acquisition and release statements
Updated Windows package lists and directory paths to new dependency package structure
Updated Python Package and Library versions
Updated Posix native files and added patches wherever necessary (Patches might need to be removed when the mirrors are updated to Python3)
Removed outdated and added new patches to target Python directory
Updated Makefiles, dependencies and added patches for specific platforms

Motivation and Context

It helps the add-on developers to code in Python3

How Has This Been Tested?

Yes, this has been tested on Jenkins and on Windows, Linux, and Android using add-ons ported to Python3

Types of change

Bug fix (non-breaking change which fixes an issue)
Improvement (non-breaking change which improves existing functionality)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

My code follows the Code guidelines of this project
My change requires a change to the documentation, either Doxygen or wiki
I have updated the documentation accordingly
I have read the CONTRIBUTING document
I have added tests to cover my change
All new and existing tests passed

Rechi · 2017-07-07T20:49:26Z

 #include "interfaces/python/AddonPythonInvoker.h"
 #include "interfaces/python/PythonInvoker.h"
-
+PyThreadState* savestate;


shouldn't this be a meber of CPythonInvoker, XBPython or something similar

Yeah, it was a temporary measure until a replacement to transfer the value of threadstate from XBPython to Pythoninvoker since paxxi said it wouldn't be a good idea to use global variables.
@Rechi

Since the functions savestate is used in don't share a common class (neither heirarchial), the option other than gloabl variable would be to define the variable static in CPythonInvoker

Rechi · 2017-07-11T15:23:37Z

+        return NULL;
+    struct module_state *st = GETSTATE(module);
+
+    st->error = PyErr_NewException((char *)"${module.@name}.Error", NULL, NULL);


this needs to be removed, I only suggested it for testing but it doesn't do what I thought

Rechi · 2017-07-12T06:27:11Z

+    else    // If pobject is of type Bytes
    {
-      buf = PyString_AsString(pObject);
+      buf = PyUnicode_AsUTF8(pObject);


I get crashes with a nullptr exception here

Rechi · 2017-07-12T06:31:17Z

      if (utf8_pyString)
      {
-        buf = PyString_AsString(utf8_pyString);
+        buf = PyUnicode_AsUTF8(utf8_pyString);


I get crashes with a nullptr exception here

arpitn30 · 2017-07-12T22:49:12Z

-        Py_DECREF(utf8_pyString);
-        return;
-      }
+      buf = PyUnicode_AsUTF8(pObject);


@Rechi Can you check if this works correctly?
According to the documentation, this function should encode the PyObject as UTF-8 and return the pointer to the string so wouldn't need another function to encode.

have tested that already during getting the crash and it worked

Rechi · 2017-07-31T07:36:51Z

+pythonmodule-pycryptodome: python36 pythonmodule-setuptools
+pythonmodule-pil: $(ZLIB) libjpeg-turbo libpng freetype2 python36 pythonmodule-setuptools
+pythonmodule-setuptools: python36
+>>>>>>> Changed Tools/Depends MakeFiles


rebase error

Rechi · 2017-07-31T07:38:15Z


 .installed-$(PLATFORM): $(LIBDYLIB)
 	cd $(PLATFORM); $(MAKE) install
+	cd $(PLATFORM); $(MAKE) regen-grammar


this is a build command, not an install command

* Updated all the files from old function names to new function names as given in pydocs - http://py3c.readthedocs.io/en/latest/reference.html * changed char to wchar_t * File pointer (FILE *) updated * Updated ob_type to be called by multi layered PyObject * Removed tp_compare since it's depreciated * Updated definitions for InitModule * Updated the Python3 Android Script * Updated the GIL acquisition and release statements * Updated module initialization from init<module> to PyInit_<module> and added the new modules to built-in library through PyImport_AppendInitTab * Charset Conversion From char To wchar_t * Updated Python WSGI Invoker

In Python 3.7 Py_Initialize() now acuires the GIL that leads to a deadlock with the subsequent PyEval_AcquireLock() call. This should fix xbmc#14885.

Rechi reviewed Jul 7, 2017

View reviewed changes

arpitn30 force-pushed the testpy3 branch 2 times, most recently from c52da26 to 3bfd92d Compare July 11, 2017 14:19

Rechi reviewed Jul 11, 2017

View reviewed changes

arpitn30 force-pushed the testpy3 branch 3 times, most recently from 4a2d9fb to 3f9df23 Compare July 11, 2017 15:58

Rechi reviewed Jul 12, 2017

View reviewed changes

arpitn30 commented Jul 12, 2017

View reviewed changes

arpitn30 force-pushed the testpy3 branch 9 times, most recently from eca0976 to 0d2d6ef Compare July 22, 2017 09:25

arpitn30 force-pushed the testpy3 branch 6 times, most recently from ea87963 to 31aba04 Compare July 30, 2017 23:55

Rechi reviewed Jul 31, 2017

View reviewed changes

arpitn30 force-pushed the testpy3 branch 3 times, most recently from 1b2fb4e to f340592 Compare August 2, 2017 06:46

arpitn30 force-pushed the testpy3 branch 10 times, most recently from 1137526 to 8e23580 Compare August 25, 2017 12:57

Rechi and others added 20 commits March 5, 2019 22:00

Revert "Reusepython"

fd42c01

[cleanup] remove unused functions

bc1d8ab

[cleanup] remove unused parameters from executeScript function

87abedc

Converted Virtual to Pure Virtual Functions

75c7571

Updated Windows Dependencies

ff8bc57

[depends] build expat-native as PIC

34a4738

[depends] add native libffi

195f9e6

[depends] Native Files Updated

e946d8e

[depends] Updated Python3 Makefile and Patches

0cc529c

[depends] Updated the PythonPath in target directories

a0ad19e

Added IOS and OSX Fix

d2668c7

Added Android Fix

8dbac6f

Added Python Version Varaiable

7f0b28a

[depends][python3] enable blake2 module

468b9c6

use python3 migration addon repo

f2397e1

[debian][cpack] switch to python3 depends

fa61dac

Check GIL state before calling PyEval_AcquireLock() to avoid a deadlock

ad1ccff

In Python 3.7 Py_Initialize() now acuires the GIL that leads to a deadlock with the subsequent PyEval_AcquireLock() call. This should fix xbmc#14885.

[TEMP] add python BASE_URL

e1687f6

[TEMP] add windows precompiled python lib

66746cb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python3 Open PR#5

Python3 Open PR#5
arpitn30 wants to merge 20 commits intomasterfrom
testpy3

arpitn30 commented Jul 7, 2017 •

edited

Loading

Uh oh!

Rechi Jul 7, 2017

Uh oh!

arpitn30 Jul 7, 2017 •

edited

Loading

Uh oh!

arpitn30 Jul 7, 2017

Uh oh!

Rechi Jul 11, 2017

Uh oh!

Rechi Jul 12, 2017

Uh oh!

Rechi Jul 12, 2017

Uh oh!

arpitn30 Jul 12, 2017

Uh oh!

Rechi Jul 13, 2017

Uh oh!

Rechi Jul 31, 2017

Uh oh!

Rechi Jul 31, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Conversation

arpitn30 commented Jul 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

How Has This Been Tested?

Types of change

Checklist:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arpitn30 Jul 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

arpitn30 commented Jul 7, 2017 •

edited

Loading

arpitn30 Jul 7, 2017 •

edited

Loading