Merge pull request #14 from Dynatrace/release_130

Prepare release v1.3.0

Author: z1c0

README.md

@@ -19,14 +19,14 @@ This SDK enables Dynatrace customers to extend request level visibility into Pyt
   * [Incoming web requests](#incoming-web-requests)
   * [Outgoing web requests](#outgoing-web-requests)
   * [Trace in-process asynchronous execution](#trace-in-process-asynchronous-execution)
-  * [Custom request attributes](#custom-request-attributes)
+  * [Custom Request Attributes](#custom-request-attributes)
   * [Custom services](#custom-services)
   * [Messaging](#messaging)
-    * [Outgoing Messages](#outgoing-messaging)
-    * [Incoming Messages](#incoming-messaging)
+    + [Outgoing Messages](#outgoing-messages)
+    + [Incoming Messages](#incoming-messages)
+- [Using the OneAgent SDK for Python with forked child processes (only available on Linux)](#using-the-oneagent-sdk-for-python-with-forked-child-processes-only-available-on-linux)
 - [Troubleshooting](#troubleshooting)
-  * [Installation issues](#installation-issues)
-  * [Post-installation issues](#post-installation-issues)
+  * [Extended SDK State](#extended-sdk-state)
 - [Repository contents](#repository-contents)
 - [Help & Support](#help--support)
   * [Read the manual](#read-the-manual)
@@ -40,12 +40,14 @@ This SDK enables Dynatrace customers to extend request level visibility into Pyt
 ## Requirements
 
 The SDK supports Python 2 ≥ 2.7 and Python 3 ≥ 3.4. Only the official CPython (that is, the "normal" Python, i.e. the Python implementation
-from <https://python.org>) is supported and only on Linux (musl libc is currently not supported) and Windows with the x86 (including
-x86-64) architecture. Additionally, `pip` ≥ 8.1.0 (2016-03-05) is required for installation.
+from <https://python.org>) is supported and only on Linux (musl libc which is used, e.g., on Alpine Linux, is currently not supported)
+and Windows with the x86 (including x86-64) architecture.
+Additionally, `pip` ≥ 8.1.0 (2016-03-05) is required for installation, and on Linux, the system should be
+[`manylinux1`-compatible](https://www.python.org/dev/peps/pep-0513/) to ensure a smooth installation via `pip`.
 
 The Dynatrace OneAgent SDK for Python is a wrapper of the [Dynatrace OneAgent SDK for C/C++](https://github.com/Dynatrace/OneAgent-SDK-for-C)
 and therefore the SDK for C/C++ is required and delivered with the Python SDK. See
-[here](https://github.com/Dynatrace/OneAgent-SDK-for-C#dynatrace-oneagent-sdk-for-cc-requirements)
+[here](https://github.com/Dynatrace/OneAgent-SDK-for-C#requirements)
 for its requirements, which also apply to the SDK for Python.
 
 The version of the SDK for C/C++ that is included in each version of the SDK for Python is shown in the following table along with the required
@@ -56,6 +58,7 @@ Dynatrace OneAgent version (it is the same as
 
 |OneAgent SDK for Python|OneAgent SDK for C/C++|Dynatrace OneAgent|Support status     |
 |:----------------------|:---------------------|:-----------------|:------------------|
+|1.3                    |1.5.1                 |≥1.179            |Supported          |
 |1.2                    |1.4.1                 |≥1.161            |Supported          |
 |1.1                    |1.3.1                 |≥1.151            |Supported          |
 |1.0                    |1.1.0                 |≥1.141            |EAP (not supported)|
@@ -429,7 +432,6 @@ the callback of a periodic timer.
 ```python
 with sdk.trace_custom_service('onTimer', 'CleanupTask'):
 	# Do the cleanup task
-	:
 ```
 
 Check out the documentation at:
@@ -535,31 +537,34 @@ See the documentation for more information:
 * [General information on tagging](https://dynatrace.github.io/OneAgent-SDK-for-Python/docs/tagging.html)
 * [Messaging tracers in the specification repository](https://github.com/Dynatrace/OneAgent-SDK#messaging)
 
-<a name="troubleshooting"></a>
-## Troubleshooting
+<a name="forking"></a>
+<a name="using-the-oneagent-sdk-for-python-with-forked-child-processes-only-available-on-linux"></a>
+## Using the OneAgent SDK for Python with forked child processes (only available on Linux)
 
-<a name="installation-issues"></a>
-### Installation issues
+Some applications, especially web servers, use a concurrency model that is based on forked child processes.
+Typically a master process is started which is responsible only for creating and managing child processes by means of forking.
+The child processes do the real work, for example handling web requests.
 
-* `ValueError` when installing, complaining about missing `DT_PYSDK_CSDK_PATH`.
+The recommended way to use the Python SDK in such a scenario is as follows: You initialize the SDK in the master process setting
+the `forkable` argument to `True`.
 
-  Make sure you are using pip to install a prebuilt package wheel for your system from PyPI, as described in [Using the OneAgent SDK for
-  Python in your application](#installation). Also make sure you are using an up-to date version of `pip`, `setuptools` and `wheel`. You can
-  try upgrading them with `python -m pip install --upgrade pip setuptools wheel` (make sure to use the same `python` that you use to install
-  the `oneagent-sdk` package). ATTENTION: If you use the system-provided pip (e.g. installed via `apt-get` on Ubuntu) you should instead use
-  a `pip` inside a `virtualenv` (the same as your project), as uprading system-provided packages via `pip` may cause issues.
+```python
+oneagent.initialize(sdk_options, forkable=True)
+```
 
-  If this does not resolve the issue, make sure you are using a supported platform, as listed in [Requirements](#requirements). If you *are*
-  using a supported system, you can try downloading the [OneAgent SDK for C/C++](https://github.com/Dynatrace/OneAgent-SDK-for-C) in the
-  version corresponding to your OneAgent SDK for Python as listed in [the table in Requirements](#requirements). Then set the
-  `DT_PYSDK_CSDK_PATH` environment variable to the `.so`/`.dll` file corresponding to your platform in the `lib` subdirectory of the C SDK
-  and retry the installation (e.g. in a bash shell, use `export DT_PYSDK_CSDK_PATH=path/to/onesdk_shared.so`). If there is no corresponding
-  directory, your platfom is not supported. Otherwise, regardless if it works with that method or not, please report an issue as desribed in
-  [Let us help you](#let-us-help-you).
+This way you will not be able to use the SDK in the master process (attempts to do so will be ignored, if applicable with
+an error code), but all forked child processes will share the same agent. This has a lower overhead, for example the
+startup of worker processes is not slowed down, and the per-worker memory overhead is reduced.
 
+For more information on forked child processes, take a look at those resources:
+* [Documentation on forking for the Dynatrace OneAgent SDK for C/C++](https://github.com/Dynatrace/OneAgent-SDK-for-C/blob/master/README.md#forking)
+* [Forking sample application](./samples/fork-sdk-sample/fork_sdk_sample.py)
+
+<a name="troubleshooting"></a>
+## Troubleshooting
 
+<a name="installation-issues"></a>
 <a name="post-installation-issues"></a>
-### Post-installation issues
 
 To debug your OneAgent SDK for Python installation, execute the following Python code:
 
@@ -580,6 +585,34 @@ Known gotchas:
   Make sure that the `pip install` or equivalent succeeded (see [here](#installation)). Also make sure you use the `pip` corresponding to your
   `python` (if in doubt, use `python -m pip` instead of `pip` for installing).
 
+* Output ending in a message like `InitResult=InitResult(status=-2, error=SDKError(-1342308345, 'Failed loading SDK stub from .../site-packages/oneagent/_impl/native/libonesdk_shared.so: "/.../libonesdk_shared.so: cannot open shared object file: No such file or directory". Check your installation of the oneagent-sdk Python package, e.g., try running `pip install --verbose --force-reinstall oneagent-sdk`.'))`.
+
+  Follow the advice of the message and run `python -m pip install --verbose --force-reinstall oneagent-sdk`
+  (or the equivalent pip invocation with the `--verbose` and `--force-reinstall` flags).
+  It is likely that you will now see another message like
+
+        ******************************************************************************
+        *** You are trying to build the Python SDK from source.                    ***
+        *** This could mean that you are using an outdated version of pip (older   ***
+        *** than 8.1.0) or you are attempting to install the SDK on an             ***
+        *** unsupported platform. Please check the requirements at                 ***
+        *** https://github.com/Dynatrace/OneAgent-SDK-for-Python#requirements      ***
+        ******************************************************************************
+
+  Make sure you are using pip to install a prebuilt package wheel for your system from PyPI, as described in [Using the OneAgent SDK for
+  Python in your application](#installation). Also make sure you are using an up-to date version of `pip`, `setuptools` and `wheel`. You can
+  try upgrading them with `python -m pip install --upgrade pip setuptools wheel` (make sure to use the same `python` that you use to install
+  the `oneagent-sdk` package). ATTENTION: If you use the system-provided pip (e.g. installed via `apt-get` on Ubuntu) you should instead use
+  a `pip` inside a `virtualenv` (the same as your project), as upgrading system-provided packages via `pip` may cause issues.
+
+  If this does not resolve the issue, make sure you are using a supported platform, as listed in [Requirements](#requirements). If you *are*
+  using a supported system, you can try downloading the [OneAgent SDK for C/C++](https://github.com/Dynatrace/OneAgent-SDK-for-C) in the
+  version corresponding to your OneAgent SDK for Python as listed in [the table in Requirements](#requirements). Then set the
+  `DT_PYSDK_CSDK_PATH` environment variable to the `.so`/`.dll` file corresponding to your platform in the `lib` subdirectory of the C SDK
+  and retry the installation (e.g. in a bash shell, use `export DT_PYSDK_CSDK_PATH=path/to/onesdk_shared.so`). If there is no corresponding
+  directory, your platform is not supported. Otherwise, regardless if it works with that method or not, please report an issue as described
+  in [Let us help you](#let-us-help-you).
+
 
 <a name="extended-sdk-state"></a>
 ### Extended SDK State

docs/conf.py

@@ -37,7 +37,7 @@
 
 import datetime
 
-from oneagent import __version__ as version
+from oneagent.version import __version__ as version
 
 # Fix spurious "= None" for instance attributes
 # https://github.com/sphinx-doc/sphinx/issues/2044

pylintrc

@@ -12,6 +12,6 @@ good-names=i,j,e,_
 max-line-length=100
 
 [TYPECHECK]
-ignored-classes=oneagent._impl.native.sdkctypesiface.SDKDllInterface
+ignored-classes=oneagent._impl.native.sdkctypesiface.SDKDllInterface,oneagent.sdk.tracers.Tracer
 redefining-builtins-modules=oneagent._impl.six.moves
-disable=missing-docstring,fixme,too-few-public-methods,missing-return-doc,useless-object-inheritance
+disable=missing-docstring,fixme,too-few-public-methods,missing-return-doc,useless-object-inheritance,cyclic-import,import-outside-toplevel

samples/fork-sdk-sample/README.md

@@ -0,0 +1,4 @@
+# OneAgent SDK for Python forking sample
+
+This example demonstrates how to use the [OneAgent SDK for
+Python](https://github.com/Dynatrace/OneAgent-SDK-for-Python) with forked child processes.

samples/fork-sdk-sample/fork_sdk_sample.py

@@ -0,0 +1,128 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright 2019 Dynatrace LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+'''This example demonstrates how to use the OneAgent SDK for Python in a
+parent/child process environment. The OneAgent SDK for Python will be initialized
+in the parent process and the child processes can then use the SDK.
+
+Note: this example will only work on Linux. There's no Windows support available.
+'''
+
+import os
+import sys
+
+import oneagent # SDK initialization functions
+import oneagent.sdk as onesdk # All other SDK functions.
+
+try: # Python 2 compatibility.
+    input = raw_input #pylint:disable=redefined-builtin
+except NameError:
+    pass
+
+
+getsdk = oneagent.get_sdk # Just to make the code shorter.
+
+
+def do_some_fancy_stuff(proc_number):
+    sdk = getsdk()
+
+    # The agent state in the child process should be ACTIVE (0).
+    print('Agent state (child process #{}): {}'.format(proc_number, sdk.agent_state), flush=True)
+
+    print('Agent found:', sdk.agent_found)
+    print('Agent is compatible:', sdk.agent_is_compatible)
+    print('Agent version:', sdk.agent_version_string)
+
+    # This call below will complete the OneAgent for Python SDK initialization and then it
+    # will start the tracer for tracing the custom service
+    with sdk.trace_custom_service('my_fancy_transaction', 'MyFancyService #{}'.format(proc_number)):
+        print('do some fancy stuff')
+
+def create_child_process(proc_number):
+    pid = os.fork()
+    if pid == 0:
+        print('child #{} is running ...'.format(proc_number))
+        do_some_fancy_stuff(proc_number)
+        print('child #{} is exiting ...'.format(proc_number))
+        sys.exit(0)
+
+    return pid
+
+def fork_children():
+    print('now starting children ...', flush=True)
+    pid_1 = create_child_process(1)
+    pid_2 = create_child_process(2)
+
+    print('waiting for child #1 ...', flush=True)
+    os.waitpid(pid_1, 0)
+    print('child #1 exited', flush=True)
+
+    print('waiting for child #2 ...', flush=True)
+    os.waitpid(pid_2, 0)
+    print('child #2 exited', flush=True)
+
+    print('all children exited', flush=True)
+
+def main():
+    # This gathers arguments prefixed with '--dt_' from sys.argv into the
+    # returned list. See also the basic-sdk-sample.
+    sdk_options = oneagent.sdkopts_from_commandline(remove=True)
+
+    # Before using the SDK you have to initialize the OneAgent. In this scenario, we
+    # initialize the SDK and prepare it for forking.
+    #
+    # Passing in the sdk_options is entirely optional and usually not required
+    # as all settings will be automatically provided by the Dynatrace OneAgent
+    # that is installed on the host.
+    #
+    # To activate the forking support add the optional 'forkable' parameter and set it to True.
+    #
+    # If you run this example on Windows then you'll get an "Invalid Argument" error back
+    # because there's no forking support for Windows available.
+    init_result = oneagent.initialize(sdk_options, forkable=True)
+    try:
+        if init_result.error is not None:
+            print('Error during SDK initialization:', init_result.error)
+
+        # While not by much, it is a bit faster to cache the result of
+        # oneagent.get_sdk() instead of calling the function multiple times.
+        sdk = getsdk()
+
+        # The agent state is one of the integers in oneagent.sdk.AgentState.
+        # Since we're using the 'forkable' mode the state will be TEMPORARILY_INACTIVE (1) on Linux.
+        print('Agent state (parent process):', sdk.agent_state)
+
+        # The instance attribute 'agent_found' indicates whether an agent could be found or not.
+        print('Agent found:', sdk.agent_found)
+
+        # If an agent was found but it is incompatible with this version of the SDK for Python
+        # then 'agent_is_compatible' would be set to false.
+        print('Agent is compatible:', sdk.agent_is_compatible)
+
+        # The agent version is a string holding both the OneAgent version and the
+        # OneAgent SDK for C/C++ version separated by a '/'.
+        print('Agent version:', sdk.agent_version_string)
+
+        if init_result.error is None:
+            fork_children()
+            input('Now wait until the path appears in the UI ...')
+    finally:
+        shutdown_error = oneagent.shutdown()
+        if shutdown_error:
+            print('Error shutting down SDK:', shutdown_error)
+
+if __name__ == '__main__':
+    main()

samples/fork-sdk-sample/setup.py

@@ -0,0 +1,55 @@
+#!/usr/bin/env python3
+#
+# Copyright 2018 Dynatrace LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import io
+
+from setuptools import setup
+
+with io.open('README.md', encoding='utf-8') as readmefile:
+    long_description = readmefile.read()
+del readmefile
+
+setup(
+    py_modules=['fork_sdk_sample'],
+    zip_safe=True,
+    name='oneagent-sdk-fork-sample',
+    version='0.0', # This sample is not separately versioned
+
+    install_requires=['oneagent-sdk==1.*,>=1.3'],
+
+    description='OneAgent SDK for Python: Fork sample application',
+    long_description=long_description,
+    long_description_content_type='text/markdown',
+    url='https://github.com/Dynatrace/OneAgent-SDK-for-Python',
+    maintainer='Dynatrace LLC',
+    maintainer_email='dynatrace.oneagent.sdk@dynatrace.com',
+    license='Apache License 2.0',
+    entry_points={
+        'console_scripts': ['oneagent-sdk-basic-sample=fork_sdk_sample:main'],
+    },
+    classifiers=[
+        'Intended Audience :: Developers',
+        'License :: OSI Approved',
+        'License :: OSI Approved :: Apache Software License', # 2.0
+        'Programming Language :: Python',
+        'Programming Language :: Python :: 2',
+        'Programming Language :: Python :: 2.7',
+        'Programming Language :: Python :: 3',
+        'Programming Language :: Python :: Implementation :: CPython',
+        'Operating System :: POSIX :: Linux',
+        'Operating System :: Microsoft :: Windows',
+        'Topic :: System :: Monitoring'
+    ])