Trygve Wastvedt

In our recent Dynamo 3.4 announcement we mentioned the availability of a new package, “PythonNet3”, which provides a new engine for Dynamo’s Python nodes (“Python Script” and “Python Script from String”). This post is a deep-dive into that terse announcement, exploring what the new engine does differently, how to transition, and where we go from here. We’re going into all the hairy detail here, and we’ll assume some knowledge of Dynamo and Python. If you need a refresher, the Primer is a great place to start.

The purpose of the new engine is to close the gap that has existed for some time between what we are able to do in the IronPython engines and the current state with the default CPython3 engine. This first release doesn’t yet close the gap, but it’s a step in that direction, with more to come!

Before we get started, we want to add a special thank you to Cyril Poupin (blog, forum), a long-time Dynamo expert, crack Python coder, blogger, and active community member. He has generously shared his in-depth knowledge of Python with our team throughout our work on the new engine, and has spent significant time testing the package and providing feedback. Thank you!

Edits

2025-02-03: Version 1.1.0 – simplify inheriting from .NET

Contents

The Landscape

We’ll start with an overview of the context in which this new package sits, as this informs subsequent discussion of new features and caveats. If you’re already familiar with IronPython, Python.NET, and the current landscape of Dynamo Python engines, feel free to skip to What’s New!

IronPython vs Python.NET

How hard can it be? You just run a Python script and return the result! And yes, if all we wanted to do was send text and numbers back and forth from Dynamo to our Python scripts, we wouldn’t need all this fuss. But we want to do a lot more than that. We want to send complex objects and geometry back and forth and we want to use Python to define logic for Dynamo’s hosts to use, for example to define filtering logic for a Revit element collector. This is hard to support because Dynamo and its hosts exist in a separate world from the world Python code typically exists in. Dynamo runs inside of the .NET runtime environment, while Python is typically interpreted and executed by CPython, the reference implementation of the Python language, written in C. This means Dynamo and Python have very different ways of representing objects and functions and, except in very simple cases, converting between the two is hard. There are two main solutions out there to deal with this gap:

IronPython: There is no gap

If Python code ran in the same place as Dynamo code, there wouldn’t be any gap to worry about. This is the approach taken by IronPython, an open-source library originally developed by Microsoft. It (re)implements the whole Python language in C# (a .NET language) so that our Python scripts are interpreted and executed in the same place Dynamo, Revit, etc. are executing. This makes it very smooth for our Python code to talk with Dynamo and its hosts, and is one reason why support for very .NET/Windows-y things like WPF and COM is so good in IronPython. However, there are some drawbacks:

  • IronPython reimplements CPython and thus needs to redo all the work CPython did. Sixteen years after the first release of Python 3, they still haven’t finished their implementation. (A comment on the size of the task, not on the valiant efforts of the open-source community!)
  • Since CPython is written in C, and C typically runs faster than Python, many Python libraries are written in C or C++. These libraries don’t work with IronPython. This includes popular libraries such as numpy and pandas.

Python.NET: Bridging the gap

Instead of reimplementing CPython, Python.NET, another open-source library, builds a bridge between the worlds of .NET and CPython. Python.NET consists of a .NET library for communicating with an instance of CPython and a Python library for communicating back with the .NET environment. There shouldn’t be any question of Python scripts or libraries not working in Dynamo-Python.NET land, since they’re running in CPython. But communication with .NET is harder because Python.NET has to do the work of translating objects and methods between the two environments.

While it’s not perfect, we believe Python.NET is the better path going forward. It’s a thinner slice of customization to link .NET and Python, and allows us to immediately benefit from work done in the massive ecosystem that is Python and its world of packages.

Dynamo Python Packages

Armed with the context of what IronPython and Python.NET are, let’s take a look at the various Python engines in Dynamo. We have (now) four different Python engines (that we know of!):

IronPython2

Built into Dynamo pre-2.8. Now a package on the package manager.

The OG! Life was pretty good back in the days of Python 2. Unfortunately, those days are long gone. Python 2 hasn’t received security updates in 15 years, which is kind of a long time for interested folks to find ways to mess with it.

IronPython3

Package on the package manager.

The logical next step, in many ways. But the IronPython 3 extension is unofficial because of the drawbacks to IronPython above.

CPython3

Built into Dynamo starting in 2.7. Default Python engine starting in 2.8.

Instead, our next engine, CPython3, switched to use Python.NET. The CPython3 engine uses Python.NET version 2.5. Not to be confused with the very old Python 2.5 – Python.NET 2.5 supports Python 3 – but it does have some impactful limitations around .NET interfaces, WPF, COM, and others.

PythonNet3

A hero arises! PythonNet3 is based, as might be guessed, on Python.NET version 3 (a major version change because of breaking changes in Python.NET itself, not because of changes in Python). Version 3 of the Python.NET library brings many improvements and optimizations that we’ve been looking forward to making use of for a while now.

Why a New Engine?

Credit: xkcd

One engine to deprecate them all! But seriously, here are the reasons why we need a new engine:

Package all the things

Extracting core functionality into packages brings several advantages. One of the biggest is that they’re updateable and removable independent of Dynamo. Now that we have Python in a package, we can deliver updates as often as we have them and many users can get those updates immediately without needing to wait for the next release of Revit/Civil 3D/etc. This also means that users of current versions of Dynamo can stay up-to-date with the latest Python functionality even if they aren’t able to update to new versions of Dynamo. We could make a package of the CPython3 engine, but we thought that would be too confusing in versions of Dynamo where CPython3 is already built-in.

Preserve existing behavior

Making a new engine, in a separate package, also allows us to leave the current CPython engine undisturbed. PythonNet3 is the future for Python in Dynamo, but we want to be certain we get that future right before we make it the default experience. We hope that separate engines will make it more clear to graph authors and users when they are using “the new thing”. Separate engines also allow side-by-side comparison of the engines in the same script.

What’s New in PythonNet3

Nice story, but what can I do with this new thing? Yep yep, here it is! The general theme, with some caveats that we’ll get into, is that CPython3 scripts should work the same in PythonNet3. There are new things, and simpler ways of doing things, in PythonNet3, but the old ways still work. In some special cases, though, behavior will change or break. We’ll highlight those as we go. Here’s a summary of the changes:

Breaking changes

Opportunities

New Package Versions

One potentially breaking change is that we’re updating the versions of the Python packages that ship with the engine. In some cases we were shipping pretty old package versions with CPython3, so this will give you access to the latest and greatest out-of-the-box. But depending on what functionality you rely on, you may need to rewrite parts of your scripts to adapt to these new versions. As with CPython3, you can install your own packages, or modify the version of these default packages, by following this documentation. Here’s a look at the version number changes:

Potentially breaking updates

Other updates

New Features

Better extension method support (LINQ!)

We have extension method support in CPython3, but only for simple cases. A number of improvements on our side and in Python.NET converged, resulting in much more robust support. Most .NET extension methods should now work, including all the goodies in LINQ. Here’s an example listing the names of trees in a Revit model:

import clr
import System
clr.AddReference('RevitAPI')
from Autodesk.Revit.DB import *
clr.AddReference('RevitServices')
from RevitServices.Persistence import DocumentManager
clr.ImportExtensions(System.Linq)

doc = DocumentManager.Instance.CurrentDBDocument
OUT = FilteredElementCollector(doc).OfClass(FamilyInstance)\
    .WhereElementIsNotElementType()\
    .Where(System.Func[Element, bool](lambda e: "Tree" in e.Name))\
    .Select[Element, str](System.Func[Element, str](lambda e: e.Name))
Code language: Python (python)

There are a few remaining caveats which we’re looking into:

  • Some methods need a little typing help. Notice that we don’t need to specify types on Where (line 12 above), but we do on Select (line 13).
  • When passing a lambda as a function parameter we always need to explicitly cast it, e.g., to System.Func[<input type>, <output type>]. (This isn’t specific to extension methods, just comes up a lot with LINQ.)
  • Some extension libraries still don’t work. The only one we’re aware of is .NET’s DataTableExtensions. We’re looking into it, but if you find others that don’t work, please let us know!

Simplified inheritance from .NET classes and interfaces

The bug encountered when defining a Python class inheriting from a .NET class or implementing a .NET interface is fixed starting in PythonNet3 version 1.1.0. Inheritance now works just as you’d expect, but see the section on overriding an out parameter for an example. (Thanks to this PR to Python.Net for the method.)

Click here for the workaround, now only needed for CPython3. Thanks to Cyril Poupin, who wrote a blog post describing this solution!
import clr
clr.AddReference('RevitAPI')
from Autodesk.Revit.DB import *
clr.AddReference("RevitServices")
from RevitServices.Persistence import DocumentManager
from RevitServices.Transactions import TransactionManager

class MyFamilyLoadOptions:
  def __new__(cls, *args, **kwargs):
    cls.args = args
    # !! Modify namespace when modifying _InnerClass !!
    cls.__namespace__ = "MyFamilyLoadOptions_xpkb"
    try:
      # Has the class already been registered?
      return __import__(cls.__namespace__)._InnerClass(*cls.args)
    except ImportError as ex:
      # Must be our first time, define the class
      class _InnerClass(IFamilyLoadOptions):
        __namespace__ = cls.__namespace__

        def __init__(self):
          super().__init__() 

        def OnFamilyFound(self, familyInUse, _overwriteParameterValues):
          overwriteParameterValues = True
          return (True, overwriteParameterValues)
            
        def OnSharedFamilyFound(self, sharedFamily, familyInUse, source, _overwriteParameterValues):
          overwriteParameterValues = True      
          return (True, overwriteParameterValues)

      return _InnerClass(*cls.args)

doc = DocumentManager.Instance.CurrentDBDocument
TransactionManager.Instance.EnsureInTransaction(doc)
opts = MyFamilyLoadOptions()
loadFamily = doc.LoadFamily(IN[0], opts, None)
TransactionManager.Instance.TransactionTaskDone()

This approach creates a wrapper class which, in its constructor, defines the actual class only on first run. It then returns an instance of the actual class from __new__, essentially becoming the actual class. If we need to edit the inner class though, the wrapper won’t know to reload it. In this case we need to change the namespace of the inner class (line 12) so that the wrapper loses track of the previous version and re-registers the new inner class. Note that the namespace and class definition name is shared across all Python nodes of the same engine in the Dynamo script. To avoid gotchas, also change the namespace when duplicating a Python node!

Python.NET 3 Updates

There’s a lot in this update that we can’t take credit for! The community that builds the Python.NET library has put in a lot of work since their version 2.5 that we use in the CPython3 engine. The update to version 3 is a large breaking change. Those breaking changes are mostly behind the scenes, and shouldn’t affect most scripts, but the deeper a script gets into the Python-.NET relationship, the more likely its behavior will change when updating. We’ll highlight some notable changes and new features here. If you really want to see all the gory detail though, take a look at their changelog. Thanks to Cyril for picking out the changes that are most relevant for Python in Dynamo!

Implement or override .NET methods with an out or ref

One common desired use-case for Python in Dynamo is to define custom logic for use in the Revit or Civil 3D API, for example a custom method for filtering elements. This is typically done by implementing a .NET interface from the Revit API using a Python class, and then passing an instance of that Python class to a Revit API method call. In IronPython this worked great, but in Python.NET there have historically been a few issues. One narrow issue is that in CPython3 we can’t implement methods with `out` or `ref` parameters. In Python.NET 3, now we can! Here’s an example of defining a custom Revit family loader:

import clr
clr.AddReference('RevitAPI')
from Autodesk.Revit.DB import *
clr.AddReference("RevitServices")
from RevitServices.Persistence import DocumentManager
from RevitServices.Transactions import TransactionManager

class MyFamilyLoadOptions(IFamilyLoadOptions):
    __namespace__ = "ns"
    def __init__(self):
      super().__init__() 

    def OnFamilyFound(self, familyInUse, _overwriteParameterValues):
      overwriteParameterValues = True
      return (True, overwriteParameterValues)
            
    def OnSharedFamilyFound(self, sharedFamily, familyInUse, source, _overwriteParameterValues):
      overwriteParameterValues = True      
      return (True, overwriteParameterValues)

doc = DocumentManager.Instance.CurrentDBDocument
TransactionManager.Instance.EnsureInTransaction(doc)
opts = MyFamilyLoadOptions()
loadFamily = doc.LoadFamily(IN[0], opts, None)
TransactionManager.Instance.TransactionTaskDone()
Code language: Python (python)

The signature of OnFamilyFound in .NET is:

bool OnFamilyFound(bool familyInUse, out bool overwriteParameterValues)Code language: C# (cs)

In Python, to implement this method we return a tuple with the method return value as the first item and values for out or ref parameters as subsequent items, ordered as they are in the method’s signature. Note that we don’t actually need the overwriteParameterValues variable; the method could be a one-liner: return (True, True). But that’s pretty cryptic, and we’d rather be kind to those that come after us, ourselves included!

Improved support for generic method overloading

In .NET it is possible to define overloaded class methods – methods which all have the same name but different numbers or types of parameters. The compiler chooses which version of the method to use based on the parameters that have been passed. When we call an overloaded .NET method from Python, it’s up to Python.NET to handle this choosing logic, something it didn’t do so well in version 2.5 (CPython3). In version 3 it’s much better! Here’s an example showing a Revit dialog where the overloaded TaskDialog.Show method works as expected:

import clr
clr.AddReference('RevitAPIUI')
from Autodesk.Revit.UI import (TaskDialog,  TaskDialogCommonButtons)

dialog = TaskDialog('Well Hello There')
dialog.CommonButtons = TaskDialogCommonButtons.Ok | TaskDialogCommonButtons.Yes
OUT = dialog.Show()Code language: Python (python)

We can use overloaded methods in CPython3 by using .NET reflection to manually pick the correct method. This still works in PythonNet3, so this isn’t a breaking new feature, but any time you see System.Reflection in your CPython3 scripts you might be able to remove it in PythonNet3.

Use overloaded C# operators

In C#, classes and structs can overload binary and unary operators. An example from the Revit API is the XYZ 3D point:

new XYZ(10, 10, 0) + new XYZ(1, 1, 1)Code language: C# (cs)

.NET knows how to add the two points because the XYZ class defines the + operator (among others). With CPython3 it wasn’t possible to use these overloads from Python. In Python.NET 3 support for this was added! The following now works as expected in Dynamo with PythonNet3:

import clr
clr.AddReference('RevitAPI')
from Autodesk.Revit.DB import XYZ

OUT = XYZ(10, 10, 0) + XYZ(1, 1, 1)Code language: Python (python)

Breaking: overriding .NET class’s constructor

When writing a Python class which inherits from a .NET class and overriding the constructor (__init__), Python.NET 3 no longer calls the base class’s __init__ for us. This gives us more flexibility on exactly when we call the base constructor, but when converting from CPython3 we will need to add this in with super().__init__().

from System.Collections.Generic import List

class MyList(List[str]):
  def __init__(self):
    super().__init__() # <- Add this in PythonNet3
    self.AddRange(["a", "b"])
    
l = MyList()
OUT = l[0]
Code language: Python (python)

Breaking: explicit enum conversion

In CPython3 enumeration objects from .NET are automatically converted to their integer equivalent in Python. This is occasionally convenient, but makes some things harder, or impossible. Now .NET enums remain .NET enums, and if we want to get their integer value we can do so explicitly, just like in .NET.

import clr
clr.AddReference('RevitAPI')
from Autodesk.Revit.DB import BuiltInCategory

c = BuiltInCategory.OST_BridgeTowers
c2 = BuiltInCategory(-2006132)
OUT = (c, int(c), c2)Code language: Python (python)
.NET enums are no longer converted to integers. Creation from integer is now possible.

Proper iterable detection

Python.NET 3 brings a small but impactful bug fix regarding iterables. In Python.NET 2.5 (as seen in the CPython3 engine), all .NET classes are considered iterable. This makes it harder to figure out whether a .NET object is a list or a single value. One common scenario where this has been annoying is when operating on input data from a host app. When working with selections or filters, often if there’s only a single piece of data it comes in as a single object, and if there are multiple pieces of data they come in as a list. This is nice when we know we’re dealing with just a single item, but if we want to handle both cases we probably want to convert that single item into a list of length one. Easy enough in Python, right?

toList = lambda x : x if hasattr(x, '__iter__') else [x]
inputList = toList(IN[0])Code language: Python (python)

But due to the bug mentioned above, in CPython3 that hasattr(x, '__iter__') check, testing whether x is a list (iterable), always returns true. In Python.NET 3 now it doesn’t! Here’s the difference in Dynamo on the selection of a family instance. The CPython3 version doesn’t create an enclosing list, while the PythonNet3 version does.

Using proper iterable detection to format data.

Migrating

Migrating from CPython3 to PythonNet3

First of all, migrating scripts from CPython3 to PythonNet3 is not a requirement. CPython3 will remain the default Python engine in Dynamo for at least the rest of 2025. But if you want to take advantage of the new goodies or be proactive, here are some thoughts and tips.

  • Take advantage of the new (in the Dynamo 3.3 release) bulk Python engine switcher (Edit > Set Python Engine) and see what happens!
  • CPython3 and PythonNet3 can coexist in separate nodes within the same graph. Copy-paste a CPython3 node, switch to PythonNet3, and compare the behavior side-by-side.

Migrating from IronPython to PythonNet3

If you don’t have a specific reason to be using IronPython in Dynamo, try switching to PythonNet3 – you might like it! If you do have a specific reason though, you might still want to stay for a bit. Here are the reasons we know of that people use IronPython instead of CPython3 (essentially a summary of the to-dos in the What’s Next section!):

  • .NET Interface implementation – Fixed in PythonNet3 1.1.0!
  • WPF – It’s not actually that bad. See below for some tips.
  • COM – Yep, stay where you are! We’ll be thinking more about COM soon.

If you do decide to make the move, see the previous section on migrating from CPython3 for some tips.

What’s Next for the Team

We’re not done yet! Our goal is to make PythonNet3 comparable to IronPython in features and ease of use. Here’s what’s on our radar going forward.

WPF

Using WPF with an MVVM pattern to display a custom window is possible in PythonNet3, and just a bit nicer than in CPython3, but not much has changed so it currently takes more effort compared to IronPython. Here’s an example with the best approach we know of at the moment, thanks to (you guessed it!) Cyril:

import clr
import System

clr.AddReference("System.Xml")
clr.AddReference("PresentationFramework")
clr.AddReference("PresentationCore")
clr.AddReference("System.Windows")
clr.AddReference("DynamoCore")
from System.Windows import LogicalTreeHelper
from Dynamo.Core import NotificationObject

class ViewModel:
  def __new__(cls, *args):
    cls.args = args
    # !! Modify namespace when modifying _ViewModel !!
    cls.__namespace__ = "ViewModel_zlcg"
    try:
      return __import__(cls.__namespace__)._ViewModel(*cls.args)
    except ImportError:
      class _ViewModel(NotificationObject):
        __namespace__ = cls.__namespace__
        def __init__(self):
          super().__init__()
        
        _text = "Hieee"  
        def get_Text(self):
          return self._text
        def set_Text(self, value):
          self._text = value
          self.RaisePropertyChanged("Text")
        Text = clr.clrproperty(str, get_Text, set_Text)
          
    return _ViewModel(*cls.args)
  
class MyWindow(System.Windows.Window):
  xaml = '''
    <Window
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Height="82" Width="350">
      <StackPanel Orientation="Horizontal" Margin="10">
        <Label Content="Text:"/>
        <Label Content="{Binding Text}"/>
        <TextBox Margin="10 0" Width="150" 
            Text="{Binding Text, UpdateSourceTrigger=PropertyChanged}" />
        <Button x:Name="Button" Content="Reverse" />
      </StackPanel>
    </Window>'''
  
  def __new__(cls):
    reader = System.Xml.XmlReader.Create(System.IO.StringReader(cls.xaml))
    window = System.Windows.Markup.XamlReader.Load(reader)
    window.__class__ = cls
    return window
    
  def __init__(self):
    super().__init__()
    self.DataContext = ViewModel()
    LogicalTreeHelper.FindLogicalNode(self, "Button").Click += self.buttonClick
      
  def buttonClick(self, sender, e):
    self.DataContext.Text = self.DataContext.Text[::-1]
      
def showWindow():
  try:
    window = MyWindow()
    window.Title = "Test"
    window.ShowDialog()   
  except Exception as ex:
    print(ex)

System.Windows.Application.Current.Dispatcher.Invoke(System.Action(showWindow))
Code language: Python (python)

Some notes:

  • We’re using Dynamo’s NotificationObject which is a basic implementation of INotifyPropertyChanged (line 20).
  • Declare properties by defining a private _propName field and get_PropName and set_PropName getter and setter methods, calling RaisePropertyChanged in the setter, and then registering everything for .NET with PropName = clr.clrproperty(str, get_PropName, set_PropName) (line 31).
  • We can’t bind events to elements directly in XAML. Instead, set the x:Name property on the element and use LogicalTreeHelper.FindLogicalNode (line 59).
  • The funny business in __new__ on line 53 merges the Window object we get from parsing the XAML with our MyWindow class. This allows for line 67, where we set a property of the System.Windows.Window class directly on our MyWindow! Note that if our XAML defined something other than a Window, we’d need to modify the class we inherit from on line 35.
  • We create the showWindow method and run it on the dispatcher (line 72) to ensure we’re running on Dynamo’s UI thread. This is only necessary in Sandbox: When inside Revit or Civil 3D we already end up on the right thread so we could just run line 66-68 directly.

Interaction with COM APIs

PythonNet3 doesn’t contain any change from CPython3 regarding interaction with Windows’ COM APIs. COM interaction is pretty seamless in IronPython but currently requires some effort in CPython3/PythonNet3. It’s on our radar! If you have a use-case for COM APIs in Dynamo Python, we’d love to hear about it on the forum.

Known Issues

This section in particular will likely stay in flux as we (and you!) discover, and we fix, discovered bugs. Please let us know if you find something goofy so we can add it here and look into it.

DataTableExtensions

As noted above in the section on extension methods, the .NET DataTableExtensions library doesn’t work with PythonNet3. It’s the only one we know of, but it’s probably not the only one which doesn’t work, so let us know if you find issues with another!

DSCore.List.Flatten

Here’s a cool thing you can do with CPython3:

import clr
clr.AddReference('DSCoreNodes')
import DSCore

flattened = DSCore.List.Flatten(IN[0])Code language: Python (python)

Unfortunately Flatten doesn’t do any flattening in PythonNet3. It’s on the list!

Why “PythonNet3”?

We’ll end with a little addendum of sorts on the hardest problem of all, naming. What should we call the new engine? Facts/criteria:

  • Also based on CPython, but using Python 3.11 instead of 3.9
  • Also based on Python.NET, but using Python.NET 3 instead of 2.5
  • Intended to become the new standard
  • Don’t want to keep making new engines every time we have a small bump in Python or Python.NET versions
  • Keep it simple

We went around on this longer than we’d care to admit. We’re not particularly happy with the result, but it’s the best we’ve got. In the end, the term is somewhat arbitrary, and perhaps the most important aspect is that it is easy to say, write, and distinguish from the other engines. Or at least that’s what we’re telling ourselves. This is likely the biggest reason we hope this is the last Python engine – if not, we’d have to come up with another name! If you come up with something better, we’d love to know about it, but we probably won’t change anything since that would just create more confusion.

That’s all you need to know! Probably. We plan to keep this up-to-date as we learn more and continue to work on the engine. We’d love to hear your feedback, questions, or suggestions on the forum. Onward!

The Dynamo Team