Clarification on Namespace....

[fog37]

Hello Forum,

In Python, my understanding is that functions, after being created, use their own memory space in RAM to hold the values of their local variables, etc. The main code (caller program) also has its own and different memory space, etc.

Do these memory spaces represent different "namespaces"? What determines when something is inside a certain namespace or another?

For example, when we create a simple variable x=5 at the prompt, what namespace is that saved in?

Namespaces are critical in the context of local and global variables so they don't get mixed up, I would say.
Thanks as usual.
Fog37

 

[PeterDonis]

In Python, my understanding is that functions, after being created, use their own memory space in RAM to hold the values of their local variables, etc. The main code (caller program) also has its own and different memory space, etc.

All of these things, as far as RAM is concerned, are just heap memory allocated by the Python interpreter. Every Python object takes up some of that heap memory. Everything is an object in Python--not just local variables of functions, but the functions themselves. From the viewpoint of the Python interpreter, all of your code is objects stored on the heap.

So thinking of separate "memory spaces" doesn't really help much in understanding what the Python interpreter is doing with your code. It's much better to think in terms of namespaces; see below for more.

Do these memory spaces represent different "namespaces"?

No.

A namespace is just another Python object: specifically, a Python dict. The keys of the dict are the names, and the values of the dict are the objects the names point to.

What determines when something is inside a certain namespace or another?

The module or code block it is in. Each Python module has a global namespace, which stores all variables defined at the module level (which includes functions and classes defined in the module). Each code block (function or class) has its own local namespace, which stores all variables that are local to the block (which can include inner functions or classes defined inside the block).

Note that when you look at reading the values of variables already defined, things get more complicated. This FAQ entry from the Python docs gives a brief overview:

https://docs.python.org/3/faq/progr...ules-for-local-and-global-variables-in-python

And the entries for the "global" and "nonlocal" statements in the docs give some more wrinkles involved when setting the values of variables:

https://docs.python.org/3/reference/simple_stmts.html#grammar-token-global-stmt

when we create a simple variable x=5 at the prompt, what namespace is that saved in?

The global namespace of the interactive session, which the Python interpreter treats as a module.

Namespaces are critical in the context of local and global variables so they don't get mixed up

Yes.

 

[PeterDonis]

thinking of separate "memory spaces" doesn't really help much in understanding what the Python interpreter is doing with your code. It's much better to think in terms of namespaces

A short way of putting this is that "variables" in Python are not labels for memory locations, as they are in languages like C. Instead, they are namespace bindings. "Setting the value of a variable" in Python does not mean writing information to a memory location; it means binding a name to an object.

For example, the line of code x = 5 tells the interpreter to do the following:

(1) Create a new int object whose value is 5 (but that does not mean we are just setting aside a memory location storing that integer value; we are creating a Python object, which is a C structure allocated from the interpreter's heap memory, and contains more information than just the integer value--for example, it contains a pointer to the type object for the int type, which is another C structure, this one allocated in the static data segment of the interpreter since it is a built-in object).

(2) Bind the object just created to the name "x" in the current namespace.

 

[fog37]

Thanks PeterDonis.

Let me see if I got it:

• In Python, namespaces are defined by individual modules: when we create a module, there is a namespace associated to it. A module is simply a Python file and everything that we create and name inside that file (which is fundamentally a Python module) belongs to the same namespace.
• Example: I write a Python script/module and call it Calculation. Such module contains a user-defined function called sqrt(). There is already another function called sqrt() in the math module. That could be an issue but there is no issue because the two sqrt() are in different namespaces.
• When I create Python program (the main script) that makes use of the both the sqrt() functions, one belongs to the math module and the other sqrt() to the Calculation module, both functions can be used with no problem because the sqrt() functions are in two different namespaces and the interpreter knows that and knows how to handle it...

 

[PeterDonis]

In Python, namespaces are defined by individual modules

Or individual functions or classes--what I called "code blocks".

everything that we create and name inside that file (which is fundamentally a Python module) belongs to the same namespace.

No, only things that are created and named at the module level. Things that are created and named inside a function or class are in the namespace of the function or class.

Such module contains a user-defined function called sqrt(). There is already another function called sqrt() in the math module. That could be an issue but there is no issue because the two sqrt() are in different namespaces.

You are ignoring several important points.

First, what makes the two sqrt functions different is not that they are "in different namespaces"; it is that they are different objects. Functions are objects in Python (everything is an object in Python). Your user-defined sqrt function is one object. The sqrt function defined in the math module is another.

Second, it is perfectly possible for both functions to be in the same namespace--they just have to be accessible using different names. For example:

>>> from math import sqrt as math_sqrt
>>> def sqrt(x):
...     print("Square root of", x)
...
>>> sqrt(2)
Square root of 2
>>> math_sqrt(2)
1.4142135623730951

Note that here the sqrt function from the math module is now in two namespaces: the math module namespace (under the name "sqrt") and the interactive interpreter session module namespace (under the name "math_sqrt"). This illustrates the fact that in Python it is perfectly possible for the same object to have references in more than one namespace--which shows why it's not a good idea to think of objects as being "in" one particular namespace.

But--it is also possible to shoot yourself in the foot by forgetting that any given namespace can only have one object pointed to by a particular name. For example:

>>> from math import sqrt
>>> def sqrt(x):
...     print("Square root of", x)
...
>>> sqrt(2)
Square root of 2

Here your user-defined sqrt function has replaced the one imported from the math module in the current namespace. The math module one is still there; it's just not accessible because you clobbered its entry in the current namespace when you defined a user-defined function with the same name.

Besides importing with a different name, another way of avoiding the above collision is this:

>>> import math
>>> def sqrt(x):
...     print("Square root of", x)
...
>>> sqrt(2)
Square root of 2
>>> math.sqrt(2)
1.4142135623730951

Here the name "math" in the current namespace points to a module, and the sqrt function in that module appears as an attribute of the module, so we can access it that way even though we also have a function named sqrt defined in our own module.

When I create Python program (the main script) that makes use of the both the sqrt() functions, one belongs to the math module and the other sqrt() to the Calculation module, both functions can be used with no problem because the sqrt() functions are in two different namespaces and the interpreter knows that and knows how to handle it...

More precisely, you can tell the interpreter how to handle it--two ways of doing that are illustrated above. But the interpreter will not "automatically" handle it in the sense that it will not prevent you from shooting yourself in the foot, as also illustrated above.

 

[fog37]

Hello PeterDonis,

In regards to OOP and Python, Python is a multi-paradigm language in the sense that it does have to use object oriented programming only. IT should be able to use other paradigms too... However, it seems that everything in Python is an "object". I see how objects are object. But the classes they objects derive from are "classes", not object. The attributes that objects have are attributes (like variables) and not objects...Or is it ok to look at everything as an object?

Anyhow, it really seems that OOP is truly engrained in Python since everything is an object. How is it then possible to do any other type of programming that is not OOP?

Thanks

 

[PeterDonis]

it seems that everything in Python is an "object".

The term "object" here does not mean the same thing as it does in the phrase "object-oriented programming". The way the Python interpreter is programmed could be viewed as a kind of OOP, but that is OOP done in C, not Python. (Note that C is not usually considered an "object-oriented" language, so this illustrates that the term "object-oriented programming" itself does not have a single well-defined meaning.) But in any case, the underlying C programming of the interpreter is not the same as the programming you do in Python itself.

it really seems that OOP is truly engrained in Python since everything is an object.

No, it isn't. See above.

 

[fog37]

Ok, I see.

I know Java and C++ seem to "embrace" OOP.

In regards to Python, creating classes and instances from those classes is essentially programming in object-oriented style... When we create an integer variable, that is an object/instance from the integer class which we verify by typing type() in Python...

But your point is that OOP and what I am describing with my examples are not really the same thing...

 

[PeterDonis]

In regards to Python, creating classes and instances from those classes is essentially programming in object-oriented style...

Creating user-defined classes and instances of them is. More precisely, writing your entire program in terms of creating user-defined classes and instances of them. But there are plenty of ways to write programs in Python without doing that. Many programs can be written without any user-defined classes at all; many others can be written making occasional use of user-defined classes without writing the entire program in terms of them.

your point is that OOP and what I am describing with my examples are not really the same thing...

Exactly. See above.

 

[fog37]

Creating user-defined classes and instances of them is. More precisely, writing your entire program in terms of creating user-defined classes and instances of them. But there are plenty of ways to write programs in Python without doing that. Many programs can be written without any user-defined classes at all; many others can be written making occasional use of user-defined classes without writing the entire program in terms of them.
Exactly. See above.

I have reflecting on the topic. Here my thoughts:
In some programming languages like C, the concept of variable indicates something different than in Python. In C, creating a variable means creating a place in memory where the data associated to the variable is stored.
For example, ##x=5## means that the integer data type 5, which is an object like anything else in Python, is stored somewhere in RAM at a location given by ##id(x)##. The name ##x## is not the variable but the pointer/reference (like an address) to the variable itself but it is not the object itself. I am trying to think of an analogy with a physical object and its label. The object and the label are obviously different things.
So how would the analogy go in the case of a variable defined in C instead?

As far as namespaces, they are collections of names with associated objects. These "ledgers" are implemented in the form of dictionaries. In the same Python program, there can be multiple, isolated namespaces (global, local, built-in, enclosed).

As an analogy, in regards to namespace and how Windows organizes files/filenames/folders, two different files can have the same name but must be located in different folders. The same file can have different names in the same folder or it can have the same name but in different folders.

In regards to OOP and everything in Python being an object: the term "object" in Python and the fact that Python objects derive from classes makes everything very similar to how OOP works. But "object" in Python is just a way to refer to and organize entities inside Python and has little to do with OOP...

 

[PeterDonis]

In C, creating a variable means creating a place in memory where the data associated to the variable is stored.

Yes. More precisely, it tells the C compiler that there will be a place in memory where that data is stored, and gives a name to that place so the compiler can understand references to it elsewhere in the code. The compiler then emits the appropriate machine code to read and write data from and to the memory location.

For example, ##x = 5## means that the integer data type 5, which is an object like anything else in Python, is stored somewhere in RAM at a location given by ##id(x)##.

You seem to be mixing C and Python here. There is no such thing as ##id(x)## in C.

I have already told you what the line of code ##x = 5## means in Python. It is not quite what you say here. First, ##x = 5## creates a namespace binding; "stored somewhere in RAM" does not really describe all that happens when that namespace binding is created. Second, while in the current CPython implementation, the ##id## function does indeed return the memory address of its argument, the Python language specification does not require that that be the case. The Python language specification does not recognize the concept of "RAM location" at all (although the Python-C API, since it involves the C language, does).

As far as namespaces, they are collections of names with associated objects. These "ledgers" are implemented in the form of dictionaries. In the same Python program, there can be multiple, isolated namespaces (global, local, built-in, enclosed).

Basically yes, although I'm not sure what you mean by "enclosed".

the term "object" in Python and the fact that Python objects derive from classes makes everything very similar to how OOP works.

Sort of. See below.

But "object" in Python is just a way to refer to and organize entities inside Python and has little to do with OOP...

The term "object" in Python does not have one single meaning. There are at least three:

(1) Individual instances, such as the int object that gets created by the line of code ##x = 5##.

(2) The C structures that the interpreter allocates to store data pertaining to individual instances.

(3) The Python type object.

How much "OOP" is involved varies depending on which meaning you intend.