Develop Reference

How to make PyLance parse NumPy style docstring properly in VSCode?

Consider the following function
def add(x: int, y: int) -> int:
""" Add two numbers
Parameters
----------
x : int
First integer
y : int
Second integer
Returns
int
Results
"""
return x + y
When I hover my mouse over add I get the following (LHS using Jedi and RHS using PyLance)
Is there a known way to get PyLance parse the docstring properly like Jedi does?

According to your description, I reproduced this problem.
Thank you for posting this problem here, and we have already submitted it, let us look forward to the improvement of this function.
Github link: How to keep the content of Parameter suggestions displayed by Pylance in the original style?

Tool to compare function signature to docstring

Is there a tool that can check if the arguments listed in the docstring match the signature of the function call? It should be able to deal with numpy-style docstrings.
I am regularly using R CMD CHECK, which finds documentation/code mismatches in R and this is quite helpful. It would be very good to have something similar in Python, but I did not find anything yet.

I just created a tool to achieve this, called pydoctest.
It will attempt to infer the types in your docstrings (not just lexically compare) and report back on mismatches between number of arguments, argument-names, argument-types, return-types, (optionally) throw error if lacking docstring and more.
It currently supports google, sphinx and numpy docstring format, but can rather easily be extended with other formats.
Example:
def func_type_mismatch(self, a: int) -> int:
"""[summary]
Args:
a (float): [description] <-- float is not int
Returns:
int: [description]
"""
pass
Running pydoctest on this function, gives this output:
Function: <function IncorrectTestClass.func_type_mismatch at 0x7f9a8b52c8c8> FAIL | Argument type differ. Argument 'a' was expected (from signature) to have type '<class 'int'>', but has (in docs) type '<class 'float'>'
Edit (June 2021): I've started the development of a vscode-extension that uses and highlights the errors.
https://marketplace.visualstudio.com/items?itemName=JeppeRask.pydoctest

I was trying to find the same, so I wrote docsig
pip install docsig
then just run docsig . and it will check this for you
/path/to/project
-----------------------
def function(✖*args) -> ✓List[str]:
"""...
:param None: ✖
:return: ✓
"""
E103: parameters missing
There are 8 other errors so far

Add return type in def line? [duplicate]

I've recently noticed something interesting when looking at Python 3.3 grammar specification:
funcdef: 'def' NAME parameters ['->' test] ':' suite
The optional 'arrow' block was absent in Python 2 and I couldn't find any information regarding its meaning in Python 3. It turns out this is correct Python and it's accepted by the interpreter:
def f(x) -> 123:
return x
I thought that this might be some kind of a precondition syntax, but:
I cannot test x here, as it is still undefined,
No matter what I put after the arrow (e.g. 2 < 1), it doesn't affect the function behavior.
Could anyone familiar with this syntax style explain it?

It's a function annotation.
In more detail, Python 2.x has docstrings, which allow you to attach a metadata string to various types of object. This is amazingly handy, so Python 3 extends the feature by allowing you to attach metadata to functions describing their parameters and return values.
There's no preconceived use case, but the PEP suggests several. One very handy one is to allow you to annotate parameters with their expected types; it would then be easy to write a decorator that verifies the annotations or coerces the arguments to the right type. Another is to allow parameter-specific documentation instead of encoding it into the docstring.

These are function annotations covered in PEP 3107. Specifically, the -> marks the return function annotation.
Examples:
def kinetic_energy(m:'in KG', v:'in M/S')->'Joules':
return 1/2*m*v**2
>>> kinetic_energy.__annotations__
{'return': 'Joules', 'v': 'in M/S', 'm': 'in KG'}
Annotations are dictionaries, so you can do this:
>>> '{:,} {}'.format(kinetic_energy(12,30),
kinetic_energy.__annotations__['return'])
'5,400.0 Joules'
You can also have a python data structure rather than just a string:
rd={'type':float,'units':'Joules',
'docstring':'Given mass and velocity returns kinetic energy in Joules'}
def f()->rd:
pass
>>> f.__annotations__['return']['type']
<class 'float'>
>>> f.__annotations__['return']['units']
'Joules'
>>> f.__annotations__['return']['docstring']
'Given mass and velocity returns kinetic energy in Joules'
Or, you can use function attributes to validate called values:
def validate(func, locals):
for var, test in func.__annotations__.items():
value = locals[var]
try:
pr=test.__name__+': '+test.__docstring__
except AttributeError:
pr=test.__name__
msg = '{}=={}; Test: {}'.format(var, value, pr)
assert test(value), msg
def between(lo, hi):
def _between(x):
return lo <= x <= hi
_between.__docstring__='must be between {} and {}'.format(lo,hi)
return _between
def f(x: between(3,10), y:lambda _y: isinstance(_y,int)):
validate(f, locals())
print(x,y)
Prints
>>> f(2,2)
AssertionError: x==2; Test: _between: must be between 3 and 10
>>> f(3,2.1)
AssertionError: y==2.1; Test: <lambda>

In the following code:
def f(x) -> int:
return int(x)
the -> int just tells that f() returns an integer (but it doesn't force the function to return an integer). It is called a return annotation, and can be accessed as f.__annotations__['return'].
Python also supports parameter annotations:
def f(x: float) -> int:
return int(x)
: float tells people who read the program (and some third-party libraries/programs, e. g. pylint) that x should be a float. It is accessed as f.__annotations__['x'], and doesn't have any meaning by itself. See the documentation for more information:
https://docs.python.org/3/reference/compound_stmts.html#function-definitions
https://www.python.org/dev/peps/pep-3107/

As other answers have stated, the -> symbol is used as part of function annotations. In more recent versions of Python >= 3.5, though, it has a defined meaning.
PEP 3107 -- Function Annotations described the specification, defining the grammar changes, the existence of func.__annotations__ in which they are stored and, the fact that it's use case is still open.
In Python 3.5 though, PEP 484 -- Type Hints attaches a single meaning to this: -> is used to indicate the type that the function returns. It also seems like this will be enforced in future versions as described in What about existing uses of annotations:
The fastest conceivable scheme would introduce silent deprecation of non-type-hint annotations in 3.6, full deprecation in 3.7, and declare type hints as the only allowed use of annotations in Python 3.8.
(Emphasis mine)
This hasn't been actually implemented as of 3.6 as far as I can tell so it might get bumped to future versions.
According to this, the example you've supplied:
def f(x) -> 123:
return x
will be forbidden in the future (and in current versions will be confusing), it would need to be changed to:
def f(x) -> int:
return x
for it to effectively describe that function f returns an object of type int.
The annotations are not used in any way by Python itself, it pretty much populates and ignores them. It's up to 3rd party libraries to work with them.

This means the type of result the function returns, but it can be None.
It is widespread in modern libraries oriented on Python 3.x.
For example, it there is in code of library pandas-profiling in many places for example:
def get_description(self) -> dict:
def get_rejected_variables(self, threshold: float = 0.9) -> list:
def to_file(self, output_file: Path or str, silent: bool = True) -> None:
"""Write the report to a file.

def f(x) -> 123:
return x
My summary:
Simply -> is introduced to get developers to optionally specify the return type of the function. See Python Enhancement Proposal 3107
This is an indication of how things may develop in future as Python is adopted extensively - an indication towards strong typing - this is my personal observation.
You can specify types for arguments as well. Specifying return type of the functions and arguments will help in reducing logical errors and improving code enhancements.
You can have expressions as return type (for both at function and parameter level) and the result of the expressions can be accessed via annotations object's 'return' attribute. annotations will be empty for the expression/return value for lambda inline functions.

def function(arg)->123:
It's simply a return type, integer in this case doesn't matter which number you write.
like Java :
public int function(int args){...}
But for Python (how Jim Fasarakis Hilliard said) the return type it's just an hint, so it's suggest the return but allow anyway to return other type like a string..

def f(x) -> str:
return x+4
print(f(45))
Will give the result : 49.
Or in other words '-> str' has NO effect on return type:
print(f(45).__class__)
<class 'int'>

-> is introduced in python3.
In simpler words, the content after the -> denotes the return type of the function.
The return type is optional.

It's just telling the user what it expects or return the value
funcname.__annotations__ will print the details
like
def function(name:str ,age:int) -> "printing the personal details ":
print(f"name is {name} age is {age}")
function("test",20)
print(function.__annotations__)
The Output
name is test age is 20
{'name': <class 'str'>, 'age': <class 'int'>, 'return': 'printing the personal details '}
even when you return the values it display nothing.

Please refer to the PEP3107 specification. These are function annotations. Python 2.x has docstrings. Similarly, Python 3 introduced the use of -> as function annotations. Python uses these while generating documentation.

How to document python function parameter types?

I know that the parameters can be any object but for the documentation it is quite important to specify what you would expect.
First is how to specify a parameter types like these below?
str (or use String or string?)
int
list
dict
function()
tuple
object instance of class MyClass
Second, how to specify params that can be of multiple types like a function that can handle a single parameter than can be int or str?
Please use the below example to demonstrate the syntax needed for documenting this with your proposed solution. Mind that it is desired to be able to hyperlink reference to the "Image" class from inside the documentation.
def myMethod(self, name, image):
"""
Does something ...
name String: name of the image
image Image: instance of Image Class or a string indicating the filename.
Return True if operation succeeded or False.
"""
return True
Note, you are welcome to suggest the usage of any documentation tool (sphinx, oxygen, ...) as long it is able to deal with the requirements.
Update:
It seams that there is some kind of support for documenting parameter types in doxygen in. general. The code below works but adds an annoying $ to the param name (because it was initially made for php).
#param str $arg description
#param str|int $arg description

There is a better way. We use
def my_method(x, y):
"""
my_method description
#type x: int
#param x: An integer
#type y: int|string
#param y: An integer or string
#rtype: string
#return: Returns a sentence with your variables in it
"""
return "Hello World! %s, %s" % (x,y)
That's it. In the PyCharm IDE this helps a lot. It works like a charm ;-)

You need to add an exclamation mark at the start of the Python docstring for Doxygen to parse it correctly.
def myMethod(self, name, image):
"""!
Does something ...
#param name String: name of the image
#param image Image: instance of Image Class or a string indicating the filename.
#return Return True if operation succeeded or False.
"""
return True

If using Python 3, you can use the function annotations described in PEP 3107.
def compile(
source: "something compilable",
filename: "where the compilable thing comes from",
mode: "is this a single statement or a suite?"):
See also function definitions.

Figured I'd post this little tidbit here since IDEA showed me this was possible, and I was never told nor read about this.
>>> def test( arg: bool = False ) -> None: print( arg )
>>> test(10)
10
When you type test(, IDLE's doc-tip appears with (arg: bool=False) -> None Which was something I thought only Visual Studio did.
It's not exactly doxygen material, but it's good for documenting parameter-types for those using your code.

Yup, #docu is right - this is the (IMHO best) way to combine both documentation schemes more or less seamlessly. If, on the other hand, you also want to do something like putting text on the doxygen-generated index page, you would add
##
# #mainpage (Sub)Heading for the doxygen-generated index page
# Text that goes right onto the doxygen-generated index page
somewhere at the beginning of your Python code.
In other words, where doxygen does not expect Python comments, use ## to alert it that there are tags for it. Where it expects Python comments (e.g. at the beginning of functions or classes), use """!.

Doxygen is great for C++, but if you are working with mostly python code you should give sphinx a try. If you choose sphinx then all you need to do is follow pep8.

How do I specify input and output data types in python comments?

I have seen several standards for writing comments about the kind of data a function expects and returns in Python. Is there a consensus on which one is best-practice?
Is the new functionality in http://www.python.org/dev/peps/pep-3107/ something I should start using for this?

Function annotations are not for a specific use, they can be used for anything.
Tools can be written to extract information from the annotations and do anything you want, including checking types or generating documentation. But python itself does not do anything with the information. You could use to a completely different purpose, i.e. to provide a function that will be called on the parameter or to declare a string of possible return values.
Annotations can be any object:
def somefunc(param1: "string annotation",
param2: 151631,
param3: any_object): -> "some information here":
and you can retrieve the objects using:
print (somefunc.func_annotations)
{'param1': "string annotation",
'param2': 151631,
'param3': <object any_object>,
'return': "some information here"}
Use case suggestions provided by the PEP:
Providing typing information
Type checking
Let IDEs show what types a function expects and returns
Function overloading / generic functions
Foreign-language bridges
Adaptation
Predicate logic functions
Database query mapping
RPC parameter marshaling
Other information
Documentation for parameters and return values
Since function annotation syntax is too new, it is really not used for any production tools.
I suggest using other methods to document that. I use epydoc to generate my documentation, and it can read parameter typing information from docstrings:
def x_intercept(m, b):
"""
Return the x intercept of the line M{y=m*x+b}. The X{x intercept}
of a line is the point at which it crosses the x axis (M{y=0}).
This function can be used in conjuction with L{z_transform} to
find an arbitrary function's zeros.
#type m: number
#param m: The slope of the line.
#type b: number
#param b: The y intercept of the line. The X{y intercept} of a
line is the point at which it crosses the y axis (M{x=0}).
#rtype: number
#return: the x intercept of the line M{y=m*x+b}.
"""
return -b/m
This example is from epydoc's website. It can generate documentation in a variety of formats, and can generate good graphs from your classes and call profiles.

If you use epydoc to produce API documentation, you have three choices.
Epytext.
ReStructuredText, RST.
JavaDoc notation, which looks a bit like epytext.
I recommend RST because it works well with sphinx for generating overall documentation suite that includes API references. RST markup is defined here. The various epydoc fields you can specify are defined here.
Example.
def someFunction( arg1, arg2 ):
"""Returns the average of *two* (and only two) arguments.
:param arg1: a numeric value
:type arg1: **any** numeric type
:param arg2: another numeric value
:type arg2: **any** numeric type
:return: mid-point (or arithmetic mean) between two values
:rtype: numeric type compatible with the args.
"""
return (arg1+arg2)/2

Python 3.5 official typing
https://docs.python.org/3/library/typing.html
This update makes types an actual part of the language.
Example:
#!/usr/bin/env python3
from typing import List
def f(x: int, ys: List[float]) -> str:
return "abc"
# Good.
f(1, [2.0, 3.0])
# Bad.
f("abc", {}) # line 12
x = 1
x = "a" # line 15
y = [] # type: List[int]
y.append("a") # line 18
This code runs normally: Python 3.5 does not enforce typing by default.
But it can however be used by static linters to diagnoze problems, e.g.:
sudo pip3 install mypy
mypy a.py
gives:
a.py:12: error: Argument 1 to "f" has incompatible type "str"; expected "int"
a.py:12: error: Argument 2 to "f" has incompatible type Dict[<nothing>, <nothing>]; expected List[float]
a.py:15: error: Incompatible types in assignment (expression has type "str", variable has type "int")
a.py:18: error: Argument 1 to "append" of "list" has incompatible type "str"; expected "int"
Existing static analyzers like Pycharm's can already parse Sphinx documentation types, but this language update gives an official canonical way that will likely prevail.
Sphinx 1.8.2 does not seem to support it yet, but it is only a matter of time: Python 3: Sphinx doesn't show type hints correctly