The open blogging platform. Say no to algorithms and paywalls.

Better Python Singleton with a Metaclass

Are you sometimes puzzled by Python singletons, classes, and perhaps have never heard of a metaclass? What is a singleton? What’s the best and most pythonic way to implement one?

In this article, I am going to cover:

  • What is a singleton anyway and why would you want to make one?
  • What alternatives are commonly used to accomplish a similar purpose?
  • A simpler, somewhat less common, and (for me) harder to find and better way to get it done using Python’s metaclass (along with some brief discussion of what a metaclass is in the first place).

My goal is to follow the Zen of python — I’ve sought a singleton solution that is simple. The solution need not be simple under the hood, but it needs to be simple to use, and obvious in what it means and does. (Which, in my opinion, is one problem that Python’s class implementation has anyway, especially with all its dunder stuff and inheritance rules. It’s gotten clear, and precise, but it is most certainly not easily approachable for beginners. In my opinion.)

Let’s dive in.

What is a Singleton Class?

A singleton class is a class that only ever has one instance. Classes in general provide a way to encapsulate (i.e. wrap up in a nice little package) a bunch of variables (what it is) and functions (what it does) and provide a way for you to keep your code organized, manageable, easier to read, and simpler to use! When you define a new class, you’re actually defining a new type. Most often you instantiate many different instances of a class with different values, with all instances having the same functionality. Consider strings — every string has the same methods such as replace() functionality Different strings, of course, have different values.

Sometimes, you want the convenience of a class (a new type) with its nice wrapping of methods into a package but you want to make sure there’s only ever one instance — one set of state — only one of them ever comes into existence. You may want this to control access to a database, for example, so that you only have one database connection. Or to an API where you want to control or serialize the requesting of data. Or some sort of cache system where you want to temporarily store results of some operation across your entire program. A logger is another common example — you may want to write some code to facilitate logging, and also be able to customize how the logging works (by overriding its behavior) in other parts of your code. But you really only want one thing that does the writing to the log stream. In all these cases, it may really mess things up if you have two different sets of values (the object state) of these types of things around.

A singleton class is a class that guarantees only one instance ever gets created. New requests to create a new instance (via the class constructor init() in Python) return the one instance if it exists, or create a new one.

What alternatives are commonly used for Python Singletons?

Since the purpose of this article is to provide a recommended method, I’ll describe other alternatives briefly and link to a few resources.

Option 1. Using a Module

You can put all your data for a singleton (what it is) as global-level variables inside a .py file as a module, and define functions (what it does) inside the module. Then, you can access the singleton data through the defined module functions. This is a great way and is often my preferred way among the alternatives because it’s super simple. You can even help make sure only the functions are available and the data stays protected by listing the things meant to be accessed outside the module by specifying them in a list of all.

all is a list of the objects inside a module that you want to make available if you were to do “from mymodule import *” which is not generally good practice, but makes clear here what the intent is — to restrict what objects are available for import. You can put variables and functions in it. e.g the snippet below will expose the variable size, and functions make_it_big and initialize. But it won’t expose the variable bigsize_multiplier:

__all__ = [ size, make_it_big, initialize ]

size = 0

bigsize_multiplier = 50

def initialize(s):
	global size
	size = s

def make_it_big():
	global size
	size = size * bigsize_multiplier

So what’s wrong with using a module as a singleton pattern?

There’s no inheritance. Sometimes, when you want to define one of something, you may also want to define one of other things based on that something. So then the other option is to resort to using Python’s implementation of objects and inheritance: class

For me, the driving need was a database connection — where I wanted to define a database, its basic connection, some basic database-level tables and properties along with operations, and then define models that each may use different tables. I wanted to be able to treat each model separately to conduct common database operations…

This need led me to approach 2:

Option 2. Container Instances

This is probably the simplest, most straightforward way to get the above need met in some circumstances. For the above example, you can define the database and its connection in a standard class, maybe even a simple singleton module as above, and then just put a variable inside each model to work with in that connection.

Here’s a contrived example with shapes:

file shape.py:

__all__ = [make_new_shape]

shape = None

def  make_new_shape(s):
	global shape
	if shape is  None:
		# you can only make new shape (and set the value) once.
		shape = s
	return shape
from shape import make_new_shape

class MyShape:
    shaper = None  # we can store the value as a class variable

    # we could also store the value as an instance variable.
    # it makes no difference - the shape will only ever be set once.

    def __init__(self, s):
        MyShape.shaper = make_new_shape(s)

    def get(self):
        return MyShape.shaper

x = MyShape("square")
y = MyShape("circle")
print(f"x is {x.get()}  y is {y.get()}")
# that prints x is square   y is square

Instances x and y always contain the same shape variable. Works great!

Container classes work well when you only need one hierarchy of inheritance. But overriding methods in derived classes gets weird when you have some state that’s not at the top of your hierarchy that you also want to be a singleton.

It gets even weirder if you want the derived instances to also be singletons, but share the same base state. You can use something called the “borg” pattern, which I’ll mention in just a bit… but to understand that you have to understand Option 3 first.

Option 3. Singleton Class

Basically, the idea is that you have a base class called a Singleton and then your other singleton classes inherit from it.

When you call a class name to make a new instance of it — to instantiate the class type into a new object instance — Python calls the class new() function. That’s responsible for setting aside the variable space for the new instance and it then calls the class init() constructor to build out that specific instance.

So the basic idea of the singleton class is to override the new() function so that it saves the first instance it makes, and then if you try to make a new one, it just returns that original one.

class Singleton(object):
    def __new__(cls, *args, **kwargs):
        # see if the instance is already in existence. If not, make a new one.
        if not hasattr(cls, '_singleton_instance'):
            cls._singleton_instance = super(Singleton, cls).__new__(cls)
        return cls._singleton_instance


class MyCoolThing(Singleton):
    def __init__(self, name):
        if hasattr(self, '_instantiated'):
            return
        self._instantiated = True
        self.name = name


mything = MyCoolThing("fred")
print(f"mything name: {mything.name} obj id: {id(mything)}")

mythingagain = MyCoolThing('jane')
print(f"mythingagainname: {mythingagain.name} obj id: {id(mythingagain)}")

Here’s the output:

mything name: fred obj id: 2454725396848
mythingagainname: fred obj id: 2454725396848

Some observations:

  • I can control re-initialization with the test in the constructor for the _instantiated private variable. (Remember the leading underscore is just a hint to the code reader.)
  • Every call to create the object will return the same object (with the same id).

But the problem with this approach is that, first of all, you have to derive other singleton classes from the Singleton base class. This makes it really challenging and yucky if you want to use a singleton in a class hierarchy with other derived classes. You might as well use a container class.

The concept of the Singleton Class shows up in many places — I’m going to point you to some articles. They’re worth checking out for background:

Singleton class in Python 3 on Geeks for Geeks and John McDonald

And one stack overflow article that helped me understand what is going on, and while it is a bit… thick … for beginners… it mentions pros and cons, with this problem as a con… “Multiple inheritance… yuck!” And I agree. It’s not pythonic at all.

Which brings us to the borg class… Which I find a really bad solution. It’s a way to define shared state in a class hierarchy. And if you ever saw that TNG episode pair featuring the Borg, I’d really rather not meet them, you know? Perhaps it got its name from the fact that with this code pattern, every instance will be assimilated.

Here is the original article. Give it a go — if you dare.

My problem with the Singleton class, and borg patterns, is that they require me to muck up the init() constructors of my classes. My singleton init has to know to check if the thing has been instantiated yet or not. I would prefer that check to be transparent.

Then I found that stack overflow article Creating a Singleton in Python (this concept was first actually initially suggested by ChatGPT, which may or may not have “learned” it from there). Now I have a new favorite way of doing this…

And Option 4 … The Singleton Metaclass!

When I first saw this, I’d honestly never heard of metaclasses. I’ve been using Python for years. I’ve taught it for years. Never heard of it. And I may be revealing some true weaknesses in myself as a coder… there’s always more to learn! So I dove in to figure out what they are and how they work. And the aforementioned stack overflow article helped a lot.

What is a metaclass? A metaclass is a class that defines how a class works on the inside. To quote the stack overflow author “A metaclass is the class of a class; that is, a class is an instance of its metaclass.” And remember, classes are normally recipes for creating (instantiating) objects. Defining the singleton pattern in the metaclass lets us control how the instantiation itself works.

Every class has a metaclass — the default metaclass is actually “type”. That’s interesting. The object “type” is callable — it returns the metaclass type of the object that you pass in to it. You can say x = type(str) for example.

But a metaclass itself is also a class. So you can derive a metaclass from another metaclass — derive it from the “type” metaclass.

Very “meta.” Yes, I know. Hang with me here….

Where most objects that don’t derive from anything implicitly derive from the very basic “object” class — every class also has a metaclass which describes how those objects work, i.e. the metaclass, and the most basic metaclass is “type”.

In any case, we can use this “class to build classes” functionality to define a better singleton pattern. I’ve modified it a bit to allow for an extra feature — the ability to control whether or not the constructor init() should actually do anything after the first singleton instance gets made. Seems useful if you want to use a subsequent re-instantiation to reset some of the singleton’s values. That allows for “lazy” initialization when you might not yet know all of the details you need to fully get your object up and going.

While this may seem confusing, there are some real advantages to using metaclasses, so press on!

class Singleton(type):
    _instances = {}
    # we are going to redefine (override) what it means to "call" a class
    # as in ....  x = MyClass(1,2,3)
    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            # we have not every built an instance before.  Build one now.
            instance = super().__call__(*args, **kwargs)
            cls._instances[cls] = instance
        else:
            instance = cls._instances[cls]
            # here we are going to call the __init__ and maybe reinitialize.
            if hasattr(cls, '__allow_reinitialization') and
                    cls.__allow_reinitialization:
                # if the class allows reinitialization, then do it
                instance.__init__(*args, **kwargs)  # call the init again
        return instance

# and with some debugging so we can see what it does:

DEBUG = True

class Singleton(type):
    _instances = {}
    def __call__(cls, *args, **kwargs):
        if cls not in cls._instances:
            if DEBUG: print(f"Singleton __call__ {cls.__name__}")
            instance = super().__call__(*args, **kwargs)
            cls._instances[cls] = instance
        else:
            instance = cls._instances[cls]
            if hasattr(cls, '__allow_reinitialization') and
                    cls.__allow_reinitialization:
                # if the class allows reinitialization, then do it
                if DEBUG: print(f"Singleton __call__ {cls.__name__} reinit")
                instance.__init__(*args, **kwargs)  # call the init again
        if DEBUG: print(f"Singleton __call__ {cls.__name__} returning "
                        f"instance id {id(instance}")
        return instance

When a class is “called” (i.e., instantiated), the call method of its metaclass is executed. In the context of Singleton above, super().call is delegating the actual creation of the class instance to the call method of the type class.

Here’s a step-by-step breakdown of what’s happening:

  • When you create an instance of a class that uses Singleton as its metaclass, Singleton.call is invoked.
  • If the class is not in _instances, then super().call(*args, **kwargs) is called to create the instance. This refers to the call method of type, which actually creates and initializes the instance.
  • If the class defines **allow_reinitialization, which because it starts with double underscores (a dunder), this variable name gets “mangled” (yes that’s the technical term) with the actual class name where it appears, which allows each level of the hierarchy to optionally allow re… **init__() to happen.
  • The created instance is stored in the _instances dictionary, and then returned.
  • Once you instantiate a class using this singleton metaclass, init() will never be called again for it even if you use different values, unless you specify the class variable __allow_reinitialization = True in there.

Here’s how you use it.

 class BaseClass(metaclass=Singleton):

        base_value = "Base Value"
        def __init__(self, value):
            if DEBUG: print(f"Initializing {type(self).__name__} with value {value}")
            self.value = value

        def show(self):
            print(f"BaseClass: value = {self.value}, base_value = {self.base_value}")


    class DerivedClass(BaseClass):
        __allow_reinitialization = True
        base_value = "Derived Value"

        def __init__(self, value):
            if DEBUG: print(f"Initializing {type(self).__name__} with value {value}")
            self.value = value
        def show(self):
            print(f"DerivedClass: value = {self.value}, base_value = {self.base_value}")


    base1 = BaseClass('base')
    base2 = BaseClass('another_base')  # will not update the value in base1.  It's already instantiated.

    derived = DerivedClass('derived')
    derived2 = DerivedClass('derived2')  # will update the value in derived

    base1.show()
    base2.show()
    derived.show()
    derived2.show()

And here is the output… Note that because just the derived class has the flag on to allow re-initialization, the calls to the constructor for the base class are ignored.

Singleton __call__ for BaseClass setting _instances
Initializing BaseClass with value base
Singleton __call__ for BaseClass returning id 1858479587184
Singleton __call__ for BaseClass returning id 1858479587184
Singleton __call__ for DerivedClass setting _instances
Initializing DerivedClass with value derived
Singleton __call__ for DerivedClass returning id 1858479587088
Singleton __call__ for DerivedClass returning id 1858479587088
BaseClass: value = base, base_value = Base Value
BaseClass: value = base, base_value = Base Value
DerivedClass: value = derived, base_value = Derived Value
DerivedClass: value = derived, base_value = Derived Value

If we derive another class that does not set this flag, the re-initialization does not inherit:

   class DerivedDerivedClass(DerivedClass):
        # __allow_reinitialization = False
        base_value = "Derived Derived Value"

        def show(self):
            print(f"DerivedDerivedClass: value = {self.value}, base_value = {self.base_value}")
            print(f"Derived_derived_value = {self.derived_derived_value}")

        def __init__(self, value):
            print(f"DerivedDerivedClass init id {id(self)}")
            super().__init__(value + " derived")
            self.derived_derived_value = value + ' derived_derived'

And the output:

--instantiating DerivedDerivedClass with value derived4
Singleton __call__ for DerivedDerivedClass setting _instances
DerivedDerivedClass init id  1858479586944
Initializing DerivedDerivedClass with value d d 1 derived
Singleton __call__ for DerivedDerivedClass returning id  1858479586944
--Now instantiating again with d d 2
Singleton __call__ for DerivedDerivedClass returning id  1858479586944
DerivedDerivedClass: value = d d 1 derived, base_value = Derived Derived Value
Derived_derived_value = d d 1 derived_derived
DerivedDerivedClass: value = d d 1 derived, base_value = Derived Derived Value
Derived_derived_value = d d 1 derived_derived

Using a metaclass for singleton has some real advantages!

  • The constructor init only happens once, unless you want it to happen every instantiation.
  • There’s nothing at all you have to do in any of the class hierarchy for typical Singleton use other than declare (metaclass = Singleton) in the base class.
  • The pattern supports inheritance, multiple inheritance, and doesn’t interfere or obfuscate any of how that should work.

So hopefully that pattern will help you out on the hopefully rare event you should need a Singleton! And well, in any case, maybe we both know a little more about Python’s class guts.

There it is! Look at that. It’s a beautiful thing!

I write occasional articles on Medium. But only when I come up with something original and interesting. I don’t re-post stuff that I’ve read elsewhere that’s easily discoverable on a search engine. If you want to follow along, great! And in any case, keep codin’ !

Thanks for reading. Comments and feedback always welcome!

PS — the singleton decorator. There’s also a singleton class decorator as it turns out. It’s yucky. It turns your class into not-a-class-at-all! So I left it out.

(re-edited to fix a bug and make some things more clear.)




Continue Learning