PIOT-CDA-09-001-A: Create module AsyncCoapClientConnector and connect it into DeviceDataManager

[labbenchstudios]

Description

• Create a new, or edit the existing, Python module named AsyncCoapClientConnector and within the module a class with the same name. This will provide your CoAP client functionality allowing you to connect to another CoAP server.
  • NOTE: These instructions make use of the following CoAP library:
    • The aiocoap open source CoAP library, located at: aiocoap. Reference: Amsüss, Christian and Wasilak, Maciej. aiocoap: Python CoAP Library. Energy Harvesting Solutions, 2013–. http://github.com/chrysn/aiocoap/.
  • This card also references the coapthon3 library:
    • The CoAPthon3 open source CoAP library, located at: CoAPthon3. Reference: G.Tanganelli, C. Vallati, E.Mingozzi, "CoAPthon: Easy Development of CoAP-based IoT Applications with Python", IEEE World Forum on Internet of Things (WF-IoT 2015).
• Connect AsyncCoapClientConnector into DeviceDataManager.

Review the README

• Please see README.md for further information on, and use of, this content.
• License for embedded documentation and source codes: PIOT-DOC-LIC

Estimated effort may vary greatly

• The estimated level of effort for this exercise shown in the 'Estimate' section below is a very rough approximation. The actual level of effort may vary greatly depending on your development and test environment, experience with the requisite technologies, and many other factors.

IMPORTANT NOTE

• The aiocoap specific code samples provided below within this exercise have been tested in a limited environment - non-Linux environments may or may not yield the correct results. You'll also need to use pip to install LinkHeader to retrieve resource discovery results.
  • One way to install LinkHeader using pip is as follows (your environment may require a slightly different command):
  • NOTE: If using virtualenv or venv, be sure to change to activate the virtual environment before executing the command below.

pip install LinkHeader

Actions

NOTE 1: As a reminder, and as mentioned in Chapter 1 of Programming the IoT within the Application configuration section, you may want to consider setting the DEFAULT_CONFIG_FILE_NAME property in ConfigConst.py to the absolute path for PiotConfig.props. While this shouldn't be necessary, as the ConfigUtil.py module (and ConfigUtil class) will attempt to locate it automatically, this will enable both the test classes and the application to find the configuration file, as the relative path will be different for each due to the path delta between the main source branch programmingtheiot and the test branch tests.

NOTE 2: There are two CoAP library dependencies included within your project's requirements.txt file:

• An asynchronous server library using asyncio (and the aiocoap library)
• A synchronous and asynchronous library using threads (and the coapthon3 library)
• For this card (and this lab module's -A named cards, we'll focus on the asynchronous approach using asyncio. This relies upon the aiocoap library.
  • Regardless of which library you choose, the calls into the CoAP client will essentially be the same - just the name of the module will be different. For this exercise, the module name will be AsyncCoapClientConnector.py, with the class name AsyncCoapClientConnector.

NOTE 3: Regarding updates to the PiotConfig.props config file: You may need to configure your client to connect to the server's IP address. There are many reasons for this, including some that are security-related. If using aiocoap, please be sure to review the FAQ here: https://aiocoap.readthedocs.io/en/latest/faq.html

Step 1: Create the module, class, and import statements

• Within the programmingtheiot.cda.connection package, create a new (or edit the existing) Python module named AsyncCoapClientConnector.py.
• Create the class AsyncCoapClientConnector within the module.
• At the beginning of the module - before the class declaration - you'll need the following import statements (if you're using the existing template, these should already be in place for you):

import logging
import socket
import threading
import traceback

import asyncio

from aiocoap import *

import programmingtheiot.common.ConfigConst as ConfigConst

from programmingtheiot.common.ConfigUtil import ConfigUtil
from programmingtheiot.common.ResourceNameEnum import ResourceNameEnum
from programmingtheiot.common.IDataMessageListener import IDataMessageListener

from programmingtheiot.cda.connection.IRequestResponseClient import IRequestResponseClient

from programmingtheiot.data.DataUtil import DataUtil

Step 2: Create the constructor

• Add the __init__() constructor.
  • Include a parameter for dataMsgListener: IDataMessageListener = None and set a class-scoped instance of self.dataMsgListener to this parameter's reference.
  • NOTE 1: You might also want to add a new method named def setDataMessageListener(self, listener: IDataMessageListener = None): to set the IDataMessageListener reference after initialization. If you always pass the reference to DeviceDataManager to the constructor (which extends IDataMessageListener), this step is unnecessary.
  • The constructor initialization should look similar to the following:

	def __init__(self, dataMsgListener: IDataMessageListener = None):
		self.config = ConfigUtil()
		self.dataMsgListener = dataMsgListener
		self.enableConfirmedMsgs = False
		self.coapClient = None
		
		self.observeRequests = { }
		self.observeTasks = { }
		
		self.host = self.config.getProperty(ConfigConst.COAP_GATEWAY_SERVICE, ConfigConst.HOST_KEY, ConfigConst.DEFAULT_HOST)
		self.port = self.config.getInteger(ConfigConst.COAP_GATEWAY_SERVICE, ConfigConst.PORT_KEY, ConfigConst.DEFAULT_COAP_PORT)
		
		self.includeDebugLogDetail = True
		
		try:
			tmpHost = socket.gethostbyname(self.host)
			
			if tmpHost:
				self.host = tmpHost
				self.uriPath = f"coap://{self.host}:{self.port}/"
				logging.info(f"CoAP client will connect to: {self.uriPath}")
				self._initEventLoop()
			else:
				logging.error(f"Can't resolve host: {self.host}")
				raise
			
		except socket.gaierror:
			logging.error(f"Failed to resolve host: {self.host}. Check hostname in config.")
			raise

Step 3: Add helper methods for setting the listener and parsing request URI's

• Public helper methods
  • Add a setter for registering the callback type that will receive incoming messages from the CoAP server's handlers:
    • NOTE: As mentioned previously, if you always pass the data message listener reference to the constructor and don't plan to use the setter method below, you can ignore this and move to "Private helper methods:"

	def setDataMessageListener(self, listener: IDataMessageListener = None) -> bool:
		if listener:
			self.dataMsgListener = listener
			return True
		
		return False

• Add the method responsible for generating resource paths:
  • This method will be named _createResourcePath() and will be called whenever a request is being made by the client to the server. This is a convenience method used to consistently generate the full URL to be issued to the CoAP server.
  • Here's a sample implementation:

	def _createResourcePath(self, resource: ResourceNameEnum = None, name: str = None):
		resourcePath = ""
		hasResource = False
		
		if resource:
			resourcePath = resourcePath + resource.value
			hasResource = True
			
		if name:
			if hasResource:
				resourcePath = resourcePath + "/"
			
			resourcePath = resourcePath + name
		
		return resourcePath

Step 4: Add internal helper methods to process requests and responses

• Internal request / response helper methods:

  • Add the following async helper methods - they can all just implement pass for now:

    • async def _handleStartObserveRequest(self, resourcePath: str = None)
    • async def _handleStopObserveRequest(self, resourcePath: str = None, ignoreErr: bool = False)
    • async def _handleDeleteRequest(self, resourcePath: str = None, enableCON: bool = False)
    • async def _handleDiscoveryRequest(self, resourcePath: str = None, enableCON: bool = False)
    • async def _handleGetRequest(self, resourcePath: str = None, enableCON: bool = False)
    • async def _handlePostRequest(self, resourcePath: str = None, payload: str = None, enableCON: bool = False)
    • async def _handlePutRequest(self, resourcePath: str = None, payload: str = None, enableCON: bool = False)

  • Add the following internal helper methods - they can all just implement pass for now:

    • def _onDeleteResponse(self, response)
    • def _onDiscoveryResponse(self, response)
    • def _onGetResponse(self, response)
    • def _onPostResponse(self, response)
    • def _onPutResponse(self, response)

• Here's one sample implementation (specifically for _handleGetRequest()). Others should follow suit for the time being. Each method's full implementation logic will be explored in later exercises within this lab module.

	async def _handleGetRequest(self, resourcePath: str = None, enableCON: bool = False):
		pass

Step 5: Add the asyncio and client initialization logic:

• These two internal methods will be called to initialize the asyncio event loop and instantiate the aiocoap CoAP client. Since they are used together, sample implementations for both are included below:

	def _initEventLoop(self):
		def startEventLoop(loop):
			asyncio.set_event_loop(loop)
			loop.run_forever()

		self._eventLoopThread = asyncio.new_event_loop()
		self._executionThread = threading.Thread(
			target = startEventLoop, 
			args = (self._eventLoopThread,), 
			daemon = True
		)

		self._executionThread.start()

		future = asyncio.run_coroutine_threadsafe(
			self._initClientContext(), 
			self._eventLoopThread
		)

		future.result()

	async def _initClientContext(self):
		self.clientContext = await Context.create_client_context()

Step 6: Integrate with DeviceDataManager

NOTE: These activities will follow the same pattern described for integrating the MQTT client adapter within the DeviceDataManager, except they'll use the instance of AsyncCoapClientConnector. Be sure to review PIOT-CDA-06-004 for details.

• Within the config path at the CDA_HOME top level director, edit PiotConfig.props to set enableCoapClient to True.
  • Add (or edit) the enableCoapClient key / value (boolean flag) in PiotConfig.props within the ConstrainedDevice section. Be sure to set it to True
  • Enable (or disable) the CoAP client within the DeviceDataManager constructor by storing a class-scoped boolean named enableCoapClient to determine if DeviceDataManager will initialize a CoAP client or not. This will be loaded from PiotConfig.props.
• Update DeviceDataManager with a reference to AsyncCoapClientConnector. This will only need to be set (and subsequently activated as indicated below) if enableCoapClient is true. For this activity, it will need to be true of course.
  • Add the requisite import statement at the beginning of the DeviceDataManager.py module:

from programmingtheiot.cda.connection.AsyncCoapClientConnector import AsyncCoapClientConnector

• Create a class-scoped instance of AsyncCoapClientConnector within the DeviceDataManager constructor called coapClient
  • Add the following code to DeviceDataManager - within the __init__() method (constructor):

self.enableCoapClient = \
	self.configUtil.getBoolean( \
		section = ConfigConst.CONSTRAINED_DEVICE, key = ConfigConst.ENABLE_COAP_CLIENT_KEY)

if self.enableCoapClient:
	self.coapClient = AsyncCoapServerAdapter(dataMsgListener = self)

• Since the CoAP client is essentially stateless, there's nothing to start or stop, so no additional logic is needed.
  • NOTE: Eventually, you'll add logic to DeviceDataManager to check if the self.coapClient reference is not None, and use it to send data to the CoAP server.

Estimate (Small = < 2 hrs; Medium = 4 hrs; Large = 8 hrs)

• Medium

Tests

• You can use the Californium Server described in PIOT-CFG-08-001 to test your CoAP client running within the CDA.

Configure the CDA to run with CoAP client enabled

• Make sure your CDA's configuration file is updated to enable the CoAP client
  • Update the config file and start the CDA in a separate terminal (or within your IDE)
  • NOTE: The config info below is JUST AN EXAMPLE

#
# CoAP client configuration information
#
[Coap.GatewayService]
credFile       = ./cred/PiotCoapCred.props
certFile       = ./cert/PiotCoapLocalCertFile.pem
host           = localhost
port           = 5683
securePort     = 5684
enableAuth     = False
enableCrypt    = False

#
# CDA specific configuration information
#
[ConstrainedDevice]
deviceLocationID = constraineddevice001
enableSimulator  = True
enableEmulator   = False
enableSenseHAT   = False
enableMqttClient = False
enableCoapServer = False
enableCoapClient = True
enableSystemPerformance = True
enableSensing    = True
enableLogging    = True
pollCycleSecs    = 5
testGdaDataPath  = /tmp/gda-data
testCdaDataPath  = /tmp/cda-data
testEmptyApp     = False
runForever       = True

Configure and run the Californium Server

• Open a separate terminal window on your system to run the Californium Server.
  • Change directory to the Californium Tools path (INSTALL_PATH/californium.tools)
  • Run cf-server with a simple discover or get request

Run the CoAP client tests

• Foo

Results (Initial)

• You should see something similar to the following, depending on how the server is configured:
  • NOTE: The following is an example response from a DISCOVER request

Results (Upon Full Lab Module Completion)

• Once you've completed all the exercises in this lab module, you should see an output that looks similar to the following, depending on how the server is configured:
  • NOTE: The following is an example response from a DISCOVER request when using the aiocoap library.