[Final Project Idea] Dyalog Jupyter kernel new features #7

xpqz · 2025-01-10T14:31:05Z

xpqz
Jan 10, 2025

Dyalog Jupyter Projects

Introduction

Jupyter notebooks are interactive documents that combine live code, equations, visualisations, and narrative text in a single, shareable format. They allow users to create and share documents containing executable code, output results, explanatory text written in Markdown, and rich media elements - Jupyter notebooks have become the de facto standard for data science and AI development, enabling researchers and practitioners to document their work, share reproducible analyses, and combine code execution with rich explanations.

A Jupyter kernel is the computational engine that executes code contained in a Jupyter notebook. It acts as an independent process that interprets and runs code in a specific programming language (like Python, R, or Julia), manages the notebook's state and variables, and communicates results back to the notebook interface through a standardised protocol. This separation between the notebook interface and kernel allows for language-agnostic functionality while maintaining a consistent user experience.

The Dyalog Jupyter kernel implements a subset of the Jupyter protocol, enabling the creation of APL notebooks, and APL-driven Jupyter books.

Here are some examples of the kernel in use:

There are two tasks of medium difficulty that can be undertaken in this project:

Dyalog Magics

Project: Dyalog Jupyter Kernel
Size: medium-175h
Difficulty Level: medium
Project URL: https://github.com/Dyalog/dyalog-jupyter-kernel
Potential Mentors: Stefan Kruger (@xpqz), Jesús Galán López (@yiyus), Martina Crippa (@martanit), Max Sun (@MaxCan-Code), Neil Kirsopp (@cursork)
Discussions: [Final Project Idea] Dyalog Jupyter kernel new features #7

The Dyalog Kernel currently does not implement any of the Jupyter magics -- meta commands that don't get executed by the kernel itself, typically interfacing with the operating system, or with Jupyter extensions, "escaping" the Dyalog kernel in this case.

The Jupyter AI extension is an example of what can be done with magic commands. We would like to implement the magics protocol in the Dyalog Kernel, and implement some of the more commonly used magic commands on top of this. Hooking Dyalog up to the Jupyter AI magics would be a great stretch goal for this. There is a floating boundary on this -- the bulk of this will still be a Python job, but depending on how deep you go, you may want to pick up some Dyalog APL to let the backend drive some of the implementation. This is a good project for the Pythonista that has a burgeoning interest in array programming and APL.

Jeremy Howard has implemented Dyalog magics for the Python kernel -- the opposite direction: it makes it possible to execute Dyalog APL commands in a Python notebook.

Expected outcomes:

magics protocol implementation, + support for a set of magics

Skills required:

Python, interest in APL

Technologies involved:

Python, possibly JavaScript, possibly some APL

Dyalog Jupyter Widgets

Project: Dyalog Jupyter Kernel
Size: medium-175h
Difficulty Level: medium
Project URL: https://github.com/Dyalog/dyalog-jupyter-kernel
Potential Mentors: Stefan Kruger (@xpqz), Jesús Galán López (@yiyus), Martina Crippa (@martanit), Max Sun (@MaxCan-Code), Neil Kirsopp (@cursork)
Discussions: [Final Project Idea] Dyalog Jupyter kernel new features #7

Jupyter supports widgets -- simple in-cell interactive controls, such as sliders, buttons etc. The widget architecture is kernel-agnostic, and any kernel can implement access to such widgets. In this project, we'd like to implement the widget protocol for the Dyalog Jupyter kernel, so that we can show a slider that sets an APL value. This would greatly enhance exploratory programming in a Dyalog APL notebook: hook a slider up to a parameter in a model and see what the effects are without having to actually type and execute cells.

Implementing the widgets protocol needs to be done firstly in Python, to implement the callbacks, and secondly on the APL side to define ways to create the widgets.

Expected outcomes:

Widgets protocol in kernel, plus APL implementation of the widget classes

Skills required:

Python, interest in APL

Technologies involved:

Python, APL

MaxCan-Code · 2025-01-10T14:51:11Z

MaxCan-Code
Jan 10, 2025

@jph00 thanks for aplnb! Any thoughts/suggestions? What features would you like to see?

0 replies

jph00 · 2025-01-10T18:37:24Z

jph00
Jan 10, 2025

Oh cool! tbh aplnb is a extremely basic implementation -- I'd love to see an official Dyalog ipython magic to replace it. There are big benefits to using a magic instead of a kernel in many situations.

I'd also love to see https://github.com/Dyalog/pynapl get some love again. It's been a long time since anyone at dyalog worked on it, and is currently in a broken state -- but it's critical infrastructure for the vast majority of the world that spends more time in Python than APL (but still loves APL!)