Python for Dummies: what does scipy.inf really do?

[nomadreid]

First, I am not even sure this is the right forum for this question, so if anyone wants to switch it to a more appropriate one, I would be grateful.
Secondly, the question here is on one specific point of Python from a mathematical point of view. I neither know nor am learning how to program, but I am interested in one point: how Python deals with infinity. From what I understand from Googling, it uses scipy.isinf or float (inf) [or float(-inf)] either as
an overflow response, or
division by zero, or
as a large (10^309 ?) [or small, -10^309 or whatever] number in something like integration, or
as a value so that any number is less than it (or, for -inf, greater than it), or
division by infinity = 0.
Is this a fair summary for the non-programmer? Any corrections would be highly appreciated.

 

[jedishrfu]

there's a discussion here:

http://docs.scipy.org/doc/numpy/reference/generated/numpy.isfinite.html

Basically it follows an IEEE 754 standard that you could look up. Programmers have found a need to distinguish between
+infinity
-infinity
and not a number

Not a number indicates that you haven't yet initialized some value whereas + or - infinity means its a infinitely large value and the sign indicates in what direction.

You could imagine how useful these values might be in defining an integral formula in a program where the from and to ranges are from -infinity to +infinity.

 

[nomadreid]

Thanks, jedishrfu. Yes, I had come across IEEE 754 -- which is why I mentioned float(inf), but the question remains the same. I had come across numpy.isinf, but you referred me to the numpy.isfinite page -- but the question is equivalent, since one is simply the negation of the other. So, for example from the page you referred me to, "For scalar input, the result is a new boolean with value True if the input is finite" -- but how does the program know that it is finite? Does it have a dictionary for operations (finite operations, convergent series) for finiteness and/or a dictionary for operations (division by zero, or divergent series, etc.) that will give infinite results? In integration, does it rely on a dictionary of standard integrals? If there is no dictionary, then how? For cases not in a dictionary, since it cannot actually use infinity, does it simply use a large number?

 

[jedishrfu]

I think if you use the strings inf and -inf instead of numbers it will interpret them as + and - infinity

    print(float('inf'))

    print(float('-inf'))

My guess is that if the reader is reading numbers and sees the inf or -inf then it will set the value to + or - infinity whereas if it sees an actual parse-able number then it will read the number otherwise it will complain that its not a number. It might allow for NaN.

Your homework is to go and test this conjecture and report back.

 

[jedishrfu]

Here's some more info:

https://www.safaribooksonline.com/library/view/python-cookbook-3rd/9781449357337/ch03s07.html

Of particular interest is that two NaN valued numbers are NEVER equal

x=float('NaN')

y=float('NaN')

print(x==y)   ## prints False

 

[nomadreid]

Thanks again, jedishrfu. The link adds a little bit of information about the return for finite arithmetic operations that Python will give when "inf" is used. So, inf in, inf out, or inf in, nan out, etc. But "if you use the strings inf and -inf instead of numbers it will interpret them as + and - infinity" seems to be begging the question, since it does not tell me what it thinks infinity is. It can't be using a set in which there are an infinite number of elements. So, what does it do for integration, infinite series, trig, etc. if an expression is not among a list of standard expressions with known results?
By the way, I started this thread warning that I do not know how to program (or even read code), so I cannot test your conjecture.

 

[jedishrfu]

Darn you're a Python theorist.

An example of use might be the say a Gaussian bell curve distribution function where it computes the area between 'a' and 'b' and you might call like this:

a=0
b=10
area=gaussianArea(a,b)
print (area)

and then later you'd like to get the full area:

a=float('-inf')
b=float('inf')
area=gaussianArea(a,b)
print (area)

The programmer who implements the function would check:
- if a and b are infinities and immediately return the canned answer
- else if one of them were infinity use a series geared for that to get the answer
- otherwise the algorithm would go thru some numerical integration to get the answer from a to b.

In any event, the programmer is using the values to determine the best course of action and return the best result so you the applications programmer don't have to worry about it. This is all theoretical of course but you get the idea.

If you ever get around to it check out youtube for tutorials on python and checkut the iPython IDE its a fantastic way to learn python. Basically a kind of web page where you can type in a multiline snippet of code and it will execute it and display the results following your input much better than the command line.

 

[DrClaude]

But "if you use the strings inf and -inf instead of numbers it will interpret them as + and - infinity" seems to be begging the question, since it does not tell me what it thinks infinity is. It can't be using a set in which there are an infinite number of elements. So, what does it do for integration, infinite series, trig, etc. if an expression is not among a list of standard expressions with known results?

You bringing up integration and infinite series hints at the fact that there is something fundamental you are misunderstanding about what this is all about.

The context is that of the representation of real numbers in a computer, which is done using floating-point representation. You basically have a finite number of bits which you can use to represent a real number, and what the IEEE 754 standard does is define how those bit should be used to encode the real number. In that standard there is also a provision for certain arrangements of those bits to represent NaN, +inf and -inf. The standard also defines some operations that should return such values, which are all based on the manipulation of real numbers, so no integrals or infinite series. There are some trig functions that can return infinities. The standard also describes what the result of some operations with ±inf and NaN should be.

So, for example from the page you referred me to, "For scalar input, the result is a new boolean with value True if the input is finite" -- but how does the program know that it is finite?

It knows it's finite if the sequence of bits does not encode the "value" infinity, as simple as that.

 

[nomadreid]

Thank you both: jedishrfu and DrClaude. First, to make sure we don't go in the wrong direction: jedishrfu used the expression "so you the applications programmer" and "a Python theorist", and suggested ways for me to learn Python. I am not a programmer of any kind, nor do I know Python, nor am I trying to learn it. I am interested in this in the same way that Bertrand Russell, Kurt Gödel, Isaac Newton, or any other mathematician who would be raised from the grave without any knowledge of machine computing would be. The answer "It knows it's finite if the sequence of bits does not encode the 'value' infinity" appears circular -- which is not necessarily bad, but it does bring me back to my original assumption that the computer depends purely on canned results "If... then return value infinity", or as DrClaude says, "certain arrangements of those bits to represent ... inf" in some cases, and in others, such as the numerical integration that jedishrfu mentioned, use a large number for infinity. My original question was whether there was anything more to it than these two cases, and so far it seems that there isn't.

 

[DrClaude]

The answer "It knows it's finite if the sequence of bits does not encode the 'value' infinity" appears circular

To be clear, I was answering your question

but how does the program know that it is finite?

This is an operation on a single float. You could well have a function numpy.iszero, and your question would be the same, namely "but how does the program know that it is zero?" The answer is also the same: it knows it is zero if the sequence of bits encodes zero.

but it does bring me back to my original assumption that the computer depends purely on canned results

Does it have a dictionary for operations (finite operations, convergent series) for finiteness and/or a dictionary for operations (division by zero, or divergent series, etc.) that will give infinite results? In integration, does it rely on a dictionary of standard integrals? If there is no dictionary, then how? For cases not in a dictionary, since it cannot actually use infinity, does it simply use a large number?

What is important to understand here is that the number of operations that are done on a float is very limited:
binary operations: arithmetic operations, exponentiation
unary operations: square root, trig functions, exponential, absolute value, and a few others

There are very few cases where these operations lead to a result outside a given set of floating point numbers. The most common is overflow, where the result is bigger than the biggest float that can be represented with a given number of bits, in which case the result should be infinity. Operations like division by 0 or square root of a negative number should return NaN.

So in that sense, yes, the computer relies on a set of predefined cases where inf or NaN should be returned. But it makes no sense to talk about integration, since you can't "integrate" a single (or even two) floats. If you need to carry out an integration in your program, what you are doing is approximating it as a sum, so you fall back on the rules for addition. Likewise, there is no such thing as "divergent series" for a one or two floats: the computer can do a finite sum of terms, and the result will be inf if there is overflow.

 

[nomadreid]

OK, thanks, Dr. Claude. I think, with the infinite () patience of you and jedishrfu, I have a clearer idea of what is going on. Come to think of it, it's not too much different than how most people deal with infinity, I guess: the Compactness Theorem, the Löwenheim-Skolem Theorem, and so forth allow us as well to operate with finite brains. So, again many thanks.

 

[Mark44]

Thanks, jedishrfu. Yes, I had come across IEEE 754 -- which is why I mentioned float(inf), but the question remains the same. I had come across numpy.isinf, but you referred me to the numpy.isfinite page -- but the question is equivalent, since one is simply the negation of the other. So, for example from the page you referred me to, "For scalar input, the result is a new boolean with value True if the input is finite" -- but how does the program know that it is finite?

When a program (which could be written in C, C++, Fortran, Python, or other) does floating point arithmetic, the various libraries that come with the programming environment know to look at a specific register in the floating point unit (FPU) to see if the previous floating point operation was successful. Years ago (up to about the late 80s or early 90s) personal computers did not come with an FPU - you could buy one as an add-on item if you were running programs that required lots of math calculations to be done quickly. Starting with one version of the Intel 486 processor (486DX), and continuing through the Intel Pentium and later series, and the comparable AMD processors, one or more FPUs are included onboard with the CPU.

The FPU has a Status Register that keeps track of a variety of faults, such as invalid arithmetic operation, divide-by-zero, numeric overflow, loss of precision, and serveral others. Without going into too much detail, an invalid arithmetic operation could be multiplication of 0 by a number whose bit pattern indicates that it is infinity.

Does it have a dictionary for operations (finite operations, convergent series) for finiteness and/or a dictionary for operations (division by zero, or divergent series, etc.) that will give infinite results? In integration, does it rely on a dictionary of standard integrals? If there is no dictionary, then how? For cases not in a dictionary, since it cannot actually use infinity, does it simply use a large number?

The FPU has separate bit patterns for +∞, -∞, as well as +0 and -0. Doing arithmetic on either of the ∞ values results in an exception being raised.

 

[jedishrfu]

Heres the wiki discussion on IEEE 754 standard

https://en.m.wikipedia.org/wiki/IEEE_754

Of interest is the sticky flags for a calculation goes out of bounds and how code must explicitly rest them which is handled by a languages support library.

I couldn't find the actual bit pattern used for the infinities, NaN... But as i remember they were patterns that could never occur as a valid floating pt value and so wouldn't occur during a calculation. I think it something to do with leading zeros in the mantissa ie an unnormalized number.

 

[Mark44]

Heres the wiki discussion on IEEE 754 standard

https://en.m.wikipedia.org/wiki/IEEE_754

Of interest is the sticky flags for a calculation goes out of bounds and how code must explicitly rest them which is handled by a languages support library.

I couldn't find the actual bit pattern used for the infinities, NaN... But as i remember they were patterns that could never occur as a valid floating pt value and so wouldn't occur during a calculation. I think it something to do with leading zeros in the mantissa ie an unnormalized number.

From a table in the wiki page for IEEE 754-1985, single precision (32 bits) floating point numbers:
Positive infinity - sign bit 0, exponent (with bias) 1111 1111 mantissa 000 0000 0000 0000 0000 0000
Negative infinity -sign bit 1, exponent (with bias) 1111 1111 mantissa 000 0000 0000 0000 0000 0000
The only difference between the two infinities is the sign bit.

I would bet that the double precision and extended double precision representations are not much different, just more bits in each (64 and 80, respectively).