The piece of code that I have looks some what like this:
glbl_array = # a 3 Gb array
def my_func( args, def_param = glbl_array):
#do stuff on args and def_param
if __name__ == '__main__':
pool = Pool(processes=4)
pool.map(my_func, range(1000))
Is there a way to make sure (or encourage) that the different processes does not get a copy of glbl_array but shares it. If there is no way to stop the copy I will go with a memmapped array, but my access patterns are not very regular, so I expect memmapped arrays to be slower. The above seemed like the first thing to try. This is on Linux. I just wanted some advice from Stackoverflow and do not want to annoy the sysadmin. Do you think it will help if the the second parameter is a genuine immutable object like glbl_array.tostring().
You can use the shared memory stuff from multiprocessing together with Numpy fairly easily:
import multiprocessing
import ctypes
import numpy as np
shared_array_base = multiprocessing.Array(ctypes.c_double, 10*10)
shared_array = np.ctypeslib.as_array(shared_array_base.get_obj())
shared_array = shared_array.reshape(10, 10)
#-- edited 2015-05-01: the assert check below checks the wrong thing
# with recent versions of Numpy/multiprocessing. That no copy is made
# is indicated by the fact that the program prints the output shown below.
## No copy was made
##assert shared_array.base.base is shared_array_base.get_obj()
# Parallel processing
def my_func(i, def_param=shared_array):
shared_array[i,:] = i
if __name__ == '__main__':
pool = multiprocessing.Pool(processes=4)
pool.map(my_func, range(10))
print shared_array
which prints
[[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
[ 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]
[ 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]
[ 3. 3. 3. 3. 3. 3. 3. 3. 3. 3.]
[ 4. 4. 4. 4. 4. 4. 4. 4. 4. 4.]
[ 5. 5. 5. 5. 5. 5. 5. 5. 5. 5.]
[ 6. 6. 6. 6. 6. 6. 6. 6. 6. 6.]
[ 7. 7. 7. 7. 7. 7. 7. 7. 7. 7.]
[ 8. 8. 8. 8. 8. 8. 8. 8. 8. 8.]
[ 9. 9. 9. 9. 9. 9. 9. 9. 9. 9.]]
However, Linux has copy-on-write semantics on fork(), so even without using multiprocessing.Array, the data will not be copied unless it is written to.
The following code works on Win7 and Mac (maybe on linux, but not tested).
import multiprocessing
import ctypes
import numpy as np
#-- edited 2015-05-01: the assert check below checks the wrong thing
# with recent versions of Numpy/multiprocessing. That no copy is made
# is indicated by the fact that the program prints the output shown below.
## No copy was made
##assert shared_array.base.base is shared_array_base.get_obj()
shared_array = None
def init(shared_array_base):
global shared_array
shared_array = np.ctypeslib.as_array(shared_array_base.get_obj())
shared_array = shared_array.reshape(10, 10)
# Parallel processing
def my_func(i):
shared_array[i, :] = i
if __name__ == '__main__':
shared_array_base = multiprocessing.Array(ctypes.c_double, 10*10)
pool = multiprocessing.Pool(processes=4, initializer=init, initargs=(shared_array_base,))
pool.map(my_func, range(10))
shared_array = np.ctypeslib.as_array(shared_array_base.get_obj())
shared_array = shared_array.reshape(10, 10)
print shared_array
For those stuck using Windows, which does not support fork() (unless using CygWin), pv's answer does not work. Globals are not made available to child processes.
Instead, you must pass the shared memory during the initializer of the Pool/Process as such:
#! /usr/bin/python
import time
from multiprocessing import Process, Queue, Array
def f(q,a):
m = q.get()
print m
print a[0], a[1], a[2]
m = q.get()
print m
print a[0], a[1], a[2]
if __name__ == '__main__':
a = Array('B', (1, 2, 3), lock=False)
q = Queue()
p = Process(target=f, args=(q,a))
p.start()
q.put([1, 2, 3])
time.sleep(1)
a[0:3] = (4, 5, 6)
q.put([4, 5, 6])
p.join()
(it's not numpy and it's not good code but it illustrates the point ;-)
If you are looking for an option that works efficiently on Windows, and works well for irregular access patterns, branching, and other scenarios where you might need to analyze different matrices based on a combination of a shared-memory matrix and process-local data, the mathDict toolkit in the ParallelRegression package was designed to handle this exact situation.
I know, I am answering to a very old question. But the this topic does not work in Windows OS. The above answers were misleading without providing substantial proof. So I had tried following code.
# -*- coding: utf-8 -*-
from __future__ import annotations
import ctypes
import itertools
import multiprocessing
import os
import time
from concurrent.futures import ProcessPoolExecutor
import numpy as np
import numpy.typing as npt
shared_np_array_for_subprocess: npt.NDArray[np.double]
def init_processing(shared_raw_array_obj: ctypes.Array[ctypes.c_double]):
global shared_np_array_for_subprocess
#shared_np_array_for_subprocess = np.frombuffer(shared_raw_array_obj, dtype=np.double)
shared_np_array_for_subprocess = np.ctypeslib.as_array(shared_raw_array_obj)
def do_processing(i: int) -> int:
print("\n--------------->>>>>>")
print(f"[P{i}] input is {i} in process id {os.getpid()}")
print(f"[P{i}] 0th element via np access: ", shared_np_array_for_subprocess[0])
print(f"[P{i}] 1st element via np access: ", shared_np_array_for_subprocess[1])
print(f"[P{i}] NP array's base memory is: ", shared_np_array_for_subprocess.base)
np_array_addr, _ = shared_np_array_for_subprocess.__array_interface__["data"]
print(f"[P{i}] NP array obj pointing memory address is: ", hex(np_array_addr))
print("\n--------------->>>>>>")
time.sleep(3.0)
return i
if __name__ == "__main__":
shared_raw_array_obj: ctypes.Array[ctypes.c_double] = multiprocessing.RawArray(ctypes.c_double, 128) # 8B * 1MB = 8MB
# This array is malloced, 0 filled.
print("Shared Allocated Raw array: ", shared_raw_array_obj)
shared_raw_array_ptr = ctypes.addressof(shared_raw_array_obj)
print("Shared Raw Array memory address: ", hex(shared_raw_array_ptr))
# Assign data
print("Assign 0, 1 element data in Shared Raw array.")
shared_raw_array_obj[0] = 10.2346
shared_raw_array_obj[1] = 11.9876
print("0th element via ptr access: ", (ctypes.c_double).from_address(shared_raw_array_ptr).value)
print("1st element via ptr access: ", (ctypes.c_double).from_address(shared_raw_array_ptr + ctypes.sizeof(ctypes.c_double)).value)
print("Create NP array from the Shared Raw array memory")
shared_np_array: npt.NDArray[np.double] = np.frombuffer(shared_raw_array_obj, dtype=np.double)
print("0th element via np access: ", shared_np_array[0])
print("1st element via np access: ", shared_np_array[1])
print("NP array's base memory is: ", shared_np_array.base)
np_array_addr, _ = shared_np_array.__array_interface__["data"]
print("NP array obj pointing memory address is: ", hex(np_array_addr))
print("NP array , Raw array points to same memory , No copies? : ", np_array_addr == shared_raw_array_ptr)
print("Now that we have native memory based NP array , Send for multi processing.")
# results = []
with ProcessPoolExecutor(max_workers=4, initializer=init_processing, initargs=(shared_raw_array_obj,)) as process_executor:
results = process_executor.map(do_processing, range(0, 2))
print("All jobs sumitted.")
for result in results:
print(result)
print("Main process is going to shutdown.")
exit(0)
here is the sample output
Shared Allocated Raw array: <multiprocessing.sharedctypes.c_double_Array_128 object at 0x000001B8042A9E40>
Shared Raw Array memory address: 0x1b804300000
Assign 0, 1 element data in Shared Raw array.
0th element via ptr access: 10.2346
1st element via ptr access: 11.9876
Create NP array from the Shared Raw array memory
0th element via np access: 10.2346
1st element via np access: 11.9876
NP array's base memory is: <multiprocessing.sharedctypes.c_double_Array_128 object at 0x000001B8042A9E40>
NP array obj pointing memory address is: 0x1b804300000
NP array , Raw array points to same memory , No copies? : True
Now that we have native memory based NP array , Send for multi processing.
--------------->>>>>>
[P0] input is 0 in process id 21852
[P0] 0th element via np access: 10.2346
[P0] 1st element via np access: 11.9876
[P0] NP array's base memory is: <memory at 0x0000021C7ACAFF40>
[P0] NP array obj pointing memory address is: 0x21c7ad60000
--------------->>>>>>
--------------->>>>>>
[P1] input is 1 in process id 11232
[P1] 0th element via np access: 10.2346
[P1] 1st element via np access: 11.9876
[P1] NP array's base memory is: <memory at 0x0000022C7FF3FF40>
[P1] NP array obj pointing memory address is: 0x22c7fff0000
--------------->>>>>>
All jobs sumitted.
0
1
Main process is going to shutdown.
The above output is from following environment:
OS: Windows 10 20H2
Python: Python 3.9.9 (tags/v3.9.9:ccb0e6a, Nov 15 2021, 18:08:50) [MSC v.1929 64 bit (AMD64)]
You can clearly see that, The numpy's pointing memory array's different for every subprocess , Meaning memcopies are made. So in Windows OS, Subprocess does not share the underlaying memory. I do think, its due to OS protection, Processes can not refer arbitrary pointer address in memory , it will lead to memory access violations.
Related
I have given a set of array where it contains real and imaginary values . I need to separate them and print it out . but its not working out . the output gives
[3. +0.j 4.5+0.j 0. +0.j]
import numpy as np
array = np.array([3,4.5,3+5j,0])
real = np.isreal(array)
print(array[real])
img = np.iscomplex(array)
print(array[img])
Referring to numpy documentation you should do the following:
print(array.real)
print(array.imag)
I guess what you are looking is to print the real numbers if it does not have any imaginary number. If it has imaginary part, then just print the imaginary part.
import numpy as np
array = np.array([3,4.5,3+5j,0])
real = np.isreal(array)
print(array[real].real)
img = np.iscomplex(array)
print(array[img].imag)
# output
[ 3. 4.5 0. ]
[5.]
is this right?
import numpy as np
array = np.array([3,4.5,3+5j,0, 12.5])
real = np.isreal(array)
#here I check to prevent round number and just cast number like 12.0 1.0 to int
print([int(i) if str(i).rsplit(".", 1)[-1] == '0' else i for i in array[real].real ])
img = np.iscomplex(array)
print([complex(int(i.real),i.imag) for i in array[img]])
output:
[3, 4.5, 0, 12.5]
[(3+5j)]
I just append 12.5 for test to see how it's working!
I want to do essentially what this person is doing but in 3D:
How can I add small 2D array to larger array?
Currently I am generated a large empty numpy array:
BigArray = np.zeros((2048,2048,1000), np.float16)
Then I am trying to add data into this array using the method shown in the stackoverflow question I linked.
The array I want to add is a smaller numpy array of 100,100,100 dimensions and the locations are given in a list of X,Y,Z coords that define the centre. Therefore my code reads as follows:
SmallArray = np.random.random((100,100,100))
X = 1000
Y = 1000
Z = 500
BigArray([X-49:Y+50, Y-49:Y+50, Z-49:Z+50]) = SmallArray
However I am getting an error that the first colon is a syntax error and I am not sure why.
Any and all help is much appreciated, thank you.
edit: typos.
Just remove the parenthesis:
BigArray[X-49:Y+50, Y-49:Y+50, Z-49:Z+50] = SmallArray
Also you will need to fix the dimensions since as-is they do not match, i.e.:
BigArray[X-50:Y+50, Y-50:Y+50, Z-50:Z+50] = SmallArray
Edit:
Also you have couple of typos in the code (like using zeroes instead of zeros and np.random instead of np.random.random), but I assume that you are not interested in those
It is interesting for me to write an very simple similar program:
#!/usr/bin/env python3
# encoding: utf-8
#如果上述二列對調，卻使用./執行的話程式就會進入無窮迴圈
#此列就是註解
#template定義時間：2019/7/5 v0.2
import sys
print (f'Python版本號={sys.version}')
#---開始include---
import numpy as np
#---開始定義class---
#---開始定義function---
print ('sam說：---主程式開始---')
if __name__=='__main__':
big_arr1 = np.random.rand(5,5,5)
small_arr1 = np.zeros((3,3,3), np.float16)
center_x = center_y = center_z = 2
#center_x-1:center_x+2代表有center_x-1、center_x、center_x+1三個
# center_y-1:center_y+2代表有center_y-1、center_y、center_y+1三個
# center_z-1:center_z+2代表有center_z-1、center_z、center_z+1三個
big_arr1[center_x-1:center_x+2,center_y-1:center_y+2,center_z-1:center_z+2]=small_arr1
print(f'big_arr1={big_arr1}')
print ('sam說：---程式結束---')
And it shows the result:
C:\Users\sam_lin\PycharmProjects\untitled\venv\Scripts\python.exe C:/Users/sam_lin/PycharmProjects/untitled/t1.py
Python版本號=3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)]
sam說：---主程式開始---
big_arr1=[[[0.97776624 0.67305297 0.46804029 0.49918114 0.5827188 ]
[0.13722586 0.74721955 0.4252602 0.0834001 0.46595479]
[0.81273576 0.93105509 0.3550111 0.86397551 0.43013513]
[0.9162833 0.73321085 0.30299702 0.88372877 0.08543336]
[0.28145705 0.21005119 0.66785858 0.46601126 0.08057115]]
[[0.96049458 0.92891106 0.59779765 0.24660834 0.28381255]
[0.78940735 0. 0. 0. 0.79491548]
[0.65879618 0. 0. 0. 0.22152894]
[0.56392902 0. 0. 0. 0.543028 ]
[0.14708571 0.16269789 0.37162561 0.19991273 0.65671269]]
[[0.78414023 0.77322729 0.93637718 0.5180622 0.18265001]
[0.68605786 0. 0. 0. 0.50407885]
[0.98517971 0. 0. 0. 0.04644972]
[0.93118566 0. 0. 0. 0.79937072]
[0.97311219 0.47284717 0.82679315 0.76999545 0.82846718]]
[[0.0707382 0.46587191 0.89547718 0.85520751 0.62759826]
[0.4973629 0. 0. 0. 0.97606185]
[0.99327387 0. 0. 0. 0.28557434]
[0.31332736 0. 0. 0. 0.34476194]
[0.12757201 0.90843887 0.68510409 0.98262188 0.48204498]]
[[0.93682369 0.88788666 0.17488394 0.68499998 0.09336082]
[0.39821201 0.39974341 0.14220433 0.55088245 0.09949949]
[0.1348445 0.11631945 0.9976837 0.21023914 0.73043136]
[0.91621389 0.4786832 0.66673538 0.37927733 0.92164015]
[0.9259155 0.91651767 0.98076657 0.83919033 0.95723923]]]
sam說：---程式結束---
Process finished with exit code 0
I need to organized a data file with chunks of named data. Data is NUMPY arrays. But I don't want to use numpy.save or numpy.savez function, because in some cases, data have to be sent on a server over a pipe or other interface. So I want to dump numpy array into memory, zip it, and then, send it into a server.
I've tried simple pickle, like this:
try:
import cPickle as pkl
except:
import pickle as pkl
import ziplib
import numpy as np
def send_to_db(data, compress=5):
send( zlib.compress(pkl.dumps(data),compress) )
.. but this is extremely slow process.
Even with compress level 0 (without compression), the process is very slow and just because of pickling.
Is there any way to dump numpy array into string without pickle? I know that numpy allows to get buffer numpy.getbuffer, but it isn't obvious to me, how to use this dumped buffer to obtaine an array back.
You should definitely use numpy.save, you can still do it in-memory:
>>> import io
>>> import numpy as np
>>> import zlib
>>> f = io.BytesIO()
>>> arr = np.random.rand(100, 100)
>>> np.save(f, arr)
>>> compressed = zlib.compress(f.getbuffer())
And to decompress, reverse the process:
>>> np.load(io.BytesIO(zlib.decompress(compressed)))
array([[ 0.80881898, 0.50553303, 0.03859795, ..., 0.05850996,
0.9174782 , 0.48671767],
[ 0.79715979, 0.81465744, 0.93529834, ..., 0.53577085,
0.59098735, 0.22716425],
[ 0.49570713, 0.09599001, 0.74023709, ..., 0.85172897,
0.05066641, 0.10364143],
...,
[ 0.89720137, 0.60616688, 0.62966729, ..., 0.6206728 ,
0.96160519, 0.69746633],
[ 0.59276237, 0.71586014, 0.35959289, ..., 0.46977027,
0.46586237, 0.10949621],
[ 0.8075795 , 0.70107856, 0.81389246, ..., 0.92068768,
0.38013495, 0.21489793]])
>>>
Which, as you can see, matches what we saved earlier:
>>> arr
array([[ 0.80881898, 0.50553303, 0.03859795, ..., 0.05850996,
0.9174782 , 0.48671767],
[ 0.79715979, 0.81465744, 0.93529834, ..., 0.53577085,
0.59098735, 0.22716425],
[ 0.49570713, 0.09599001, 0.74023709, ..., 0.85172897,
0.05066641, 0.10364143],
...,
[ 0.89720137, 0.60616688, 0.62966729, ..., 0.6206728 ,
0.96160519, 0.69746633],
[ 0.59276237, 0.71586014, 0.35959289, ..., 0.46977027,
0.46586237, 0.10949621],
[ 0.8075795 , 0.70107856, 0.81389246, ..., 0.92068768,
0.38013495, 0.21489793]])
>>>
THe default pickle method provides a pure ascii output. To get (much) better performance, use the latest version available. Versions 2 and above are binary and, if memory serves me right, allows numpy arrays to dump their buffer directly into the stream without addtional operations.
To select version to use, add the optional argument while pickling (no need to specify it while unpickling), for instance pkl.dumps(data, 2).
To pick the latest possible version, use pkl.dumps(data, -1)
Note that if you use different python versions, you need to specify the lowest supported version.
See Pickle documentation for details on the different versions
There is a method tobytes which, according to my benchmarks is faster than other alternatives.
Take with a grain of salt, as some of my experiments may be misguided or plainly wrong, but it is a method of dumping numpy array into strings.
Keep in mind that you will need to have some additional data out of band, mainly the data type of the array and also its shape. That may be a deal breaker or it may not be rellevant. It's easy to recover the original shape by calling .fromstring(..., dtype=...).reshape(...).
Edit: A maybe incomplete example
##############
# Generation #
##############
import numpy as np
arr = np.random.randint(1, 7, (4,6))
arr_dtype = arr.dtype.str
arr_shape = arr.shape
arr_data = arr.tobytes()
# Now send / store arr_dtype, arr_shape, arr_data, where:
# arr_dtype is string
# arr_shape is tuple of integers
# arr_data is bytes
############
# Recovery #
############
arr = np.frombuffer(arr_data, dtype=arr_dtype).reshape(arr_shape)
I am not considering the column/row ordering, because I know that numpy supports things about that but I have never used it. If you want to support / need to have the memory arranged in a specific fashion --regarding row/column for multidimensional arrays-- you may need to take that into account at some point.
Also: frombuffer doesn't copy the buffer data, it creates the numpy structure as a view (maybe not exactly that, but you know what I mean). If that's undesired behaviour you can use fromstring (which is deprecated but seems to work on 1.19) or use frombuffer followed by a np.copy.
I am learning how to make my script run faster. I think parallel is a good way. So I try gevent and multiprocessing. But I am confused by it's different result. Let me show two examples I met,
ex 1:
a=np.zeros([3])
def f(i):
a[i]=1
print a
def f_para():
p=multiprocessing.Pool()
p.map(f, range(3))
def f_asy():
threads = [gevent.spawn(f, i) for i in xrange(3)]
gevent.joinall(threads)
f_para()
[ 0. 1. 0.]
[ 0. 0. 1.]
[ 1. 0. 0.]
f_asy()
[ 1. 0. 0.]
[ 1. 1. 0.]
[ 1. 1. 1.]
I find that using multiprocessing, the global object a never change in fat, and after running f_para(), a is still the original array. While running f_asy(), it's different, a changed.
ex 2:
def f2(i):
subprocess.call(['./a.out', str(i)])
time.sleep(0.2)
def f2_loop():
for i in xrange(20):
f2(i)
def f2_para():
p=multiprocessing.Pool()
p.map(f2, range(20))
def f2_asy():
threads = [gevent.spawn(f2, i) for i in xrange(20)]
gevent.joinall(threads)
%timeit -n1 f2_loop()
1 loop, best of 3: 4.22 s per loop
%timeit -n1 f2_asy()
1 loop, best of 3: 4.22 s per loop
%timeit -n1 f2_para()
1 loop, best of 3: 657 ms per loop
I find that f2_asy() don't decrease run time. And the output of f2_asy() is one by one, just as f2_loop(), so there is no parallel in f2_asy() I think.
The a.out is a simple c++ code:
#include <iostream>
int main(int argc, char* argv[])
{
std::cout<<argv[1]<<std::endl;
return 0;
}
So my question is that:
why in ex 1, f_para can change the value of global array a ?
why in ex 2, f2_asy can't do parallel?
Dose any knows the difference between gevent and multiprocessing? I am very grateful if you are willing to explain it.
ex1:
when you use multiprocess each process has separate memory(unlike threads)
ex2:
gevent does not create threads, it creates Greenlets(coroutines)!
Greenlets all run inside of the OS process for the main program but are scheduled cooperatively.
Only one greenlet is ever running at any given time.
This differs from any of the real parallelism constructs provided by multiprocessing or threading libraries which do spin processes and POSIX threads which are scheduled by the operating system and are truly parallel.
I was constructing a database for a deep learning algorithm. The points I'm interested in are these:
with open(fname, 'a+') as f:
f.write("intens: " + str(mean_intensity_this_object) + "," + "\n")
f.write("distances: " + str(dists_this_object) + "," + "\n")
Where mean_intensity_this_object is a list and dists_this_object is a numpy.array, something I didn't pay enough attention to to begin with. After I opened the file, I found out that the second variable, distances, looks very different to intens: The former is
distances: [430.17802963 315.2197058 380.33997833 387.46190951 41.93648858
221.5210474 488.99452579],
and the latter
intens: [0.15381262,..., 0.13638344],
The important bit is that the latter is a standard list, while the former is very hard to read: multiple lines without delimiters and unclear rules for starting a new line. Essentially as a result I had to rerun the whole tracking algorithm and change str(dists_this_object) to str(dists_this_object.tolist()) which increased the file size.
So, my question is: why does this happen? Is it possible to save np.array objects in a more readable format, like lists?
In an interactive Python session:
>>> import numpy as np
>>> x = np.arange(10)/.33 # make an array of floats
>>> x
array([ 0. , 3.03030303, 6.06060606, 9.09090909,
12.12121212, 15.15151515, 18.18181818, 21.21212121,
24.24242424, 27.27272727])
>>> print(x)
[ 0. 3.03030303 6.06060606 9.09090909 12.12121212
15.15151515 18.18181818 21.21212121 24.24242424 27.27272727]
>>> print(x.tolist())
[0.0, 3.0303030303030303, 6.0606060606060606, 9.09090909090909, 12.121212121212121, 15.15151515151515, 18.18181818181818, 21.21212121212121, 24.242424242424242, 27.27272727272727]
The standard display for a list is with [] and ,. The display for an array is without ,. If there are over 1000 items, the array display employs an ellipsis
>>> print(x)
[ 0. 3.03030303 6.06060606 ..., 3024.24242424
3027.27272727 3030.3030303 ]
while the list display continues to show every value.
In this line, did you add the ..., or is that part of the print?
intens: [0.15381262,..., 0.13638344],
Or doing the same with a file write:
In [299]: with open('test.txt', 'w') as f:
...: f.write('array:'+str(x)+'\n')
...: f.write('list:'+str(x.tolist())+'\n')
In [300]: cat test.txt
array:[ 0. 3.33333333 6.66666667 10. 13.33333333
16.66666667 20. 23.33333333 26.66666667 30. ]
list:[0.0, 3.3333333333333335, 6.666666666666667, 10.0, 13.333333333333334, 16.666666666666668, 20.0, 23.333333333333336, 26.666666666666668, 30.0]
np.savetxt gives more control over the formatting of an array, for example:
In [312]: np.savetxt('test.txt',[x], fmt='%10.6f',delimiter=',')
In [313]: cat test.txt
0.000000, 3.333333, 6.666667, 10.000000, 13.333333, 16.666667, 20.000000, 23.333333, 26.666667, 30.000000
The default array print is aimed mainly at interactive work, where you want to see enough of the values to see whether they are right or not, but you don't intend to reload them. The savetxt/loadtxt pair are better for that.
The savetxt does, roughly:
for row in x:
f.write(fmt%tuple(row))
where fmt is constructed from your input paramater and the number of items in the row, e.g. ', '.join(['%10.6f']*10)+'\n'
In [320]: print('[%s]'%', '.join(['%10.6f']*10)%tuple(x))
[ 0.000000, 3.333333, 6.666667, 10.000000, 13.333333, 16.666667, 20.000000, 23.333333, 26.666667, 30.000000]
Actually python converts both in the same way: str(object) calls object.__str__() or object.__repr__() if the former does not exist. From that point it is the responsibility of object to provide its string representation.
Python lists and numpy arrays are different objects, designed and implemented by different people to serve different needs so it is to be expected that their __str__ and __repr__ methods do not behave the same.