Develop Reference

ValueError: Invalid classes inferred from unique values of `y` in XGBoost

I'm new to the Data Science field and I'm trying to apply XGBoost in a table having 5 rows × 46 columns
and my last column is my target column.
import sys
!{sys.executable} -m pip install xgboost
import xgboost as xgb
from sklearn.model_selection import train_test_split
x=df_null_mean.iloc[:,:-1]
y=df_null_mean.iloc[:,-1]
x_cols=x.columns
# Splitting the dataset into the Training set and Test set
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=0)
# Fitting XGBoost to the training data
my_model = xgb.XGBClassifier()
my_model.fit(x_train, y_train)
and the error I'm getting is
ValueError: Invalid classes inferred from unique values of `y`. Expected: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53
54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89
90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107
108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125
126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143
144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161
162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179
180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197
198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215
216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233
234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251
252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269
270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287
288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305
306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323
324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341
342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359
360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377
378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395
396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413
414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431
432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449
450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467
468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485
486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503
504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521
522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539
540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557
558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575
576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593
594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611
612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629
630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647
648 649 650 651 652 653], got [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53
54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89
90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107
108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125
126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143
144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161
162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179
180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197
198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215
216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233
234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251
252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269
270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287
288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305
306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323
324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341
342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359
360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377
378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395
396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413
414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431
432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449
450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467
468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485
486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503
504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521
522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539
540 541 542 544 545 546 547 548 549 550 551 553 554 555 556 557 558 559
560 562 563 564 565 567 568 569 570 572 573 574 576 577 578 579 580 581
582 583 584 585 586 587 588 589 590 591 592 593 595 596 598 599 602 605
606 607 608 609 614 615 617 619 622 626 628 629 630 631 632 638 639 640
642 647 650 659 665 673 674 680 684 685 688 691 703 710 713 714 715 716
717 718 719 727 730 731 763 786 812 850 854 857 862 870 876 878 880 884
889 892 894 898 900 902]
Can anyone help me with the resolution?
import sys
!{sys.executable} -m pip install xgboost
import xgboost as xgb
from sklearn.model_selection import train_test_split
x=df_null_mean.iloc[:,:-1]
y=df_null_mean.iloc[:,-1]
x_cols=x.columns
# Splitting the dataset into the Training set and Test set
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=0)
# Fitting XGBoost to the training data
my_model = xgb.XGBClassifier()
my_model.fit(x_train, y_train)

I think you need to have the class numerotated from 0 to n-1 where n is your number of class.
Try this:
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
y_train = le.fit_transform(y_train)

read csv and Iterate through 10 row blocks

I am trying to read a CSV file and Iterate through 10-row blocks.
The data is quite unusual, with two columns and 10-row blocks.
57485 rows x 2 columns in the format below:
Grid-ref= 1,148
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
Grid-ref= 1,311
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
Grid-ref= 1,312
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
Every 10 rows consist of a grid reference and two records X/Y ref.
The grid reference and X value is in column 1, the Y value is in column 2, and then 9 rows with 12 columns, in column one.
The blocks (x) 0 - 9, represent months (Jan - December
The blocks (y) 0 - 9, represent years (1991-2000)
So for 0, is 1991
3020 is January 1991, 2820 is February 1991
Grid-ref = 1 (X),148 (Y)
The code below reads 10 rows, but keeps repeating the first row in all following 10-row blocks??
I don't understand why it keeps repeating the first row??
Any suggestions to resolve this would be appreciated..
## Python 3.6
## Read in the datasets (they are in CSV format)
data = pd.read_csv('cru-ts-2-10-1991-2000-cutdown.csv', skiprows=5, na_values = [-999] )
## View data >> 57485 rows x 2 columns
#print(data)
#print(len(data)) ## len = 57485
## header = pd.MultiIndex.from_product([['Grid-ref', 'Xref', 'Yref'], ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec' ] ])
# df = pd.DataFrame(np.random.randn(10, 11),
# index=['1991','1992','1993','1994','1995', '1996', '1997', '1998', '1999', '2000', '2001'],
# columns=header)
# print(data.head(10)) ## prints chunks of 10 rows
def chunker(seq, size):
return (seq[pos:pos + size] for pos in range(0, len(seq), size))
for group in chunker(data, 10):
print(group)
The first two block:
Grid-ref= 1 148
0 3020 2820 3040 2880 1740 1360 980 990 1410 ... NaN
1 3020 2820 3040 2880 1740 1360 980 990 1410 ... NaN
2 3020 2820 3040 2880 1740 1360 980 990 1410 ... NaN
3 3020 2820 3040 2880 1740 1360 980 990 1410 ... NaN
4 3020 2820 3040 2880 1740 1360 980 990 1410 ... NaN
5 3020 2820 3040 2880 1740 1360 980 990 1410 ... NaN
6 3020 2820 3040 2880 1740 1360 980 990 1410 ... NaN
7 3020 2820 3040 2880 1740 1360 980 990 1410 ... NaN
8 3020 2820 3040 2880 1740 1360 980 990 1410 ... NaN
9 3020 2820 3040 2880 1740 1360 980 990 1410 ... NaN
Grid-ref= 1 148
10 Grid-ref= 1 311.0
11 490 290 280 230 200 250 440 530 460 ... NaN
12 490 290 280 230 200 250 440 530 460 ... NaN
13 490 290 280 230 200 250 440 530 460 ... NaN
14 490 290 280 230 200 250 440 530 460 ... NaN
15 490 290 280 230 200 250 440 530 460 ... NaN
16 490 290 280 230 200 250 440 530 460 ... NaN
17 490 290 280 230 200 250 440 530 460 ... NaN
18 490 290 280 230 200 250 440 530 460 ... NaN
19 490 290 280 230 200 250 440 530 460 ... NaN

Pandas is good for uniform columnar data. If your input isn't uniform, you can preprocess it and then load the dataframe. This one is easy, all you need to do is scan for grid headers and remove them. Since the data itself is numeric, separated by whitespace, a simple split will parse it. This example creates a list but if the dataset is large, it may be reasonable to write to an intermediate file instead.
import csv
import re
import pandas as pd
grid_re = re.compile(r"Grid-ref=\s*(\d+),(\d+)")
with open('test.csv') as fobj:
table = []
try:
while True:
# find next block
for line in fobj:
match = grid_re.search(line)
if match:
grid_xy = list(match.groups())
break
else:
raise StopIteration()
for _ in range(10):
line = next(fobj)
# add row plus grid columns
table.append(line.strip().split() + grid_xy)
except StopIteration:
pass
df = pd.DataFrame(table)
print(df)

Python and Sqlite3, transforming data text file to sql database

So I've been given an assignment/Challenge to complete but I just don't know whee to start with it I've got experience with Python but not with using databases and data transformation onto the description.
So here is a snippet of my text file I've been given:
Grid-ref= 1, 148
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630
Grid-ref= 1, 311
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
490 290 280 230 200 250 440 530 460 420 530 450
Grid-ref= 1, 312
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
460 280 260 220 190 240 430 520 450 400 520 410
So from this i must then create a database containing 4 columns like so:
Xref Yref Date Value
1 148 1,1,2000 3020
1 148 1,2,2000 2820
I hope you can see the pattern so grid-ref= 1, 148 are my X & Y co-ords then each value is obviously the value but i need to iterate through and for each value it then gives it the new date which is just the 1st of each month for 10 years.
So far I have this code which i know isn't much it's a start.
import os
import csv
import sqlite3
f_path = os.path.dirname(os.path.abspath(__file__)) + "/data/"
db = sqlite3.connect('output.db')
cursor = db.cursor()
cursor.execute('CREATE TABLE Data (Xref, Yref, Date, Value)')
date = 2000 - 2010
grid = 'Xref, Yref'
with open(f_path + "data.to.use.txt") as file_read:
for row in csv.DictReader(file_read):
cursor.execute('''INSERT INTO Data
VALUES (:Xref, :Yref, :Date, :Value)''', row)
db.commit()
db.close()
Thank you for all feedback and guidance, I'm in unfamiliar territory with this type of task and hope you can help.

you could try the below code. I am not quite not clear with date requirement . So I just added a month for each entry
from datetime import date,datetime
from dateutil.relativedelta import relativedelta
Xref=''
Yref=''
date =datetime.strptime('2000-01-01', '%Y-%m-%d')
with open('C:\Users\shmathew\Desktop\Sample\sample.txt') as file_read:
for row in file_read:
print row
if 'Grid-ref' in row:
Xref = row.split(',')[0].split('= ')[1]
Yref = row.split(',')[1]
else:
for Value in row.split(' '):
date = date + relativedelta(months=+1)
print Xref.strip(),Yref.strip(),date,Value.strip()
Sample output
490 290 280 230 200 250 440 530 460 420 530 450
1 311 2009-08-01 00:00:00 490
1 311 2009-09-01 00:00:00 290
1 311 2009-10-01 00:00:00 280
1 311 2009-11-01 00:00:00 230
1 311 2009-12-01 00:00:00 200
1 311 2010-01-01 00:00:00 250
1 311 2010-02-01 00:00:00 440
1 311 2010-03-01 00:00:00 530
1 311 2010-04-01 00:00:00 460
1 311 2010-05-01 00:00:00 420
1 311 2010-06-01 00:00:00 530
1 311 2010-07-01 00:00:00 450
490 290 280 230 200 250 440 530 460 420 530 450
1 311 2010-08-01 00:00:00 490
1 311 2010-09-01 00:00:00 290
1 311 2010-10-01 00:00:00 280
1 311 2010-11-01 00:00:00 230
1 311 2010-12-01 00:00:00 200
1 311 2011-01-01 00:00:00 250
1 311 2011-02-01 00:00:00 440
1 311 2011-03-01 00:00:00 530
1 311 2011-04-01 00:00:00 460
1 311 2011-05-01 00:00:00 420
1 311 2011-06-01 00:00:00 530
1 311 2011-07-01 00:00:00 450
490 290 280 230 200 250 440 530 460 420 530 450
1 311 2011-08-01 00:00:00 490
1 311 2011-09-01 00:00:00 290
1 311 2011-10-01 00:00:00 280
1 311 2011-11-01 00:00:00 230
1 311 2011-12-01 00:00:00 200
1 311 2012-01-01 00:00:00 250
1 311 2012-02-01 00:00:00 440
1 311 2012-03-01 00:00:00 530
1 311 2012-04-01 00:00:00 460
1 311 2012-05-01 00:00:00 420
1 311 2012-06-01 00:00:00 530
1 311 2012-07-01 00:00:00 450

Python for loop over list with directories does not find every value

I have a list of directories. I try just to keep those, which are named by number and not have some string name, e.g. "lastStableBuild". The following code removes every non-digit named directory except from "lastUnsuccessfulBuild".
for dir in listdir:
print(dir, " ", end="")
if not dir.isdigit():
listdir.remove(dir)
While debugging I tried to print the whole list. There are several directories with "string names" listed, but not "lastUnsuccessfulBuild". Then I printed the whole list again, and now it was listed, and the other were removed. After executing the removal code again "lastUnsuccessfulBuild" was removed, too. I am really irritated.
This is the complete code:
listdir = os.listdir(rootdir)
for dir in listdir:
print(dir, " ", end="")
if not dir.isdigit():
listdir.remove(dir)
print("")
print("________________After first removal_____________________________")
for dir in listdir:
print(dir, " ", end="")
for dir in listdir:
if not dir.isdigit():
listdir.remove(dir)
print("")
print("________________After second removal_____________________________")
for dir in listdir:
print(dir, " ", end="")
Which produces the following output:
> python listdir.py
> 778 761 794 792 885 877 811 873 871 679
> 726 771 837 691 783 751 813 780 852 723 721 801 826
> 784 757 846 812 782 724 855 804 847 831 874 718 741 703
> 789 756 688 825 824 748 875 697 676 798 747 705 736 765
> 858 717 745 863 876 823 865 704 719 732 800 880 767 759
> 842 815 753 779 680 833 752 734 716 696 851 834 682 708
> 844 758 772 884 828 737 795 754 829 797 827 lastStableBuild
> 850 714 886 774 887 762 883 860 707 687 739 861 805 722
> 763 859 845 817 822 864 821 749 699 746 802 711 857 867
> 740 684 698 692 760 832 693 830 839 806 750 738 728 678
> 818 710 727 701 888 849 729 843 841 744 764 814 872 766
> 808 712 854 695 810 881 731 862 878 713 816 848 720 777
> 700 lastUnstableBuild 775 769 742 791 866 694 725 796 770
> 773 879 685 787 809 legacyIds 690 856 799 838 768 730 803
> 793 677 686 683 807 743 lastFailedBuild 702 870 735 715 820
> lastSuccessfulBuild 835 836 785 733 776 706 786 781 788 709
> 790 868 689 882 869 840 755
> ________________After first removal_____________________________
> 778 761 794 792 885 877 811 873 871 679 726 771 837 691 783
> 751 813 780 852 723 721 801 826 784 757 846 812 782 724
> 855 804 847 831 874 718 741 703 789 756 688 825 824 748
> 875 697 676 798 747 705 736 765 858 717 745 863 876 823
> 865 704 719 732 800 880 767 759 842 815 753 779 680 833
> 752 734 716 696 851 834 682 708 844 758 772 884 828 737
> 795 754 829 797 827 675 850 714 886 774 887 762 883 860
> 707 687 739 861 805 722 763 859 845 817 822 864 821 749
> 699 746 802 711 857 867 740 684 698 692 760 832 693 830
> 839 806 750 738 728 678 818 710 727 701 888 849 729 843
> 841 744 764 814 872 766 808 712 854 695 810 881 731 862
> 878 713 816 848 720 777 700 819 775 769 742 791 866 694
> 725 796 770 773 879 685 787 809 681 690 856 799 838 768
> 730 803 793 677 686 683 807 743 853 702 870 735 715 820
> lastUnsuccessfulBuild 835 836 785 733 776 706 786 781 788
> 709 790 868 689 882 869 840 755
> ________________After second removal_____________________________
> 778 761 794 792 885 877 811 873 871 679 726 771 837 691 783
> 751 813 780 852 723 721 801 826 784 757 846 812 782 724
> 855 804 847 831 874 718 741 703 789 756 688 825 824 748
> 875 697 676 798 747 705 736 765 858 717 745 863 876 823
> 865 704 719 732 800 880 767 759 842 815 753 779 680 833
> 752 734 716 696 851 834 682 708 844 758 772 884 828 737
> 795 754 829 797 827 675 850 714 886 774 887 762 883 860
> 707 687 739 861 805 722 763 859 845 817 822 864 821 749
> 699 746 802 711 857 867 740 684 698 692 760 832 693 830
> 839 806 750 738 728 678 818 710 727 701 888 849 729 843
> 841 744 764 814 872 766 808 712 854 695 810 881 731 862
> 878 713 816 848 720 777 700 819 775 769 742 791 866 694
> 725 796 770 773 879 685 787 809 681 690 856 799 838 768
> 730 803 793 677 686 683 807 743 853 702 870 735 715 820
> 835 836 785 733 776 706 786 781 788 709 790 868 689 882
> 869 840 755
The only thing, which comes to my mind, is that the for loop has a maximum of iterations, but I could not found anything about it.
What is the explanation of this behavior? And is there a better solution to get just the directories with "number names". (I can't determine the number of directories.)

Well, don't change a list while iterating over it.
Use list comprehension:
listdir = [dir for dir in listdir if dir.isdigit()]

file output in python giving me garbage

When I write the following code I get garbage for an output. It is just a simple program to find prime numbers. It works when the first for loops range only goes up to 1000 but once the range becomes large the program fail's to output meaningful data
output = open("output.dat", 'w')
for i in range(2, 10000):
prime = 1
for j in range(2, i-1):
if i%j == 0:
prime = 0
j = i-1
if prime == 1:
output.write(str(i) + " " )
output.close()
print "writing finished"

This is a known Notepad bug. Check out
http://blogs.msdn.com/oldnewthing/archive/2007/04/17/2158334.aspx
The classic way to trigger this bug is to put "Bush hid the facts" in a file, save it, reopen it, and scream about conspiracy theories, but I guess "2 3 5 7 11 13 17" works too, except that you don't get to scream about conspiracy theories.

You're setting a single variable named prime ten thousand times to 1, then 9998 times possibly setting it to 0, and finally (if it's not been set to 0) outputting one incomplete line (no line-end). I suspect that's not what you want to do! Maybe something like...:
output = open("output.dat", 'w')
for i in range(2, 10000):
prime = 1
for j in range(2, i-1):
if i%j == 0:
prime = 0
break
if prime == 1:
output.write(str(i) + " " )
output.close()
print "writing finished"
Note the very different indentation from what you had posted. I also used break to break out of an inner loop, which I think was what you meant where you wrote j = i - 1 (which would in fact have absolutely no effect since j would just be set to its next natural value in the very next leg of that inner loop, which would still run to the end).

With fixed indentation (which I'll have to assume is a bad paste job, otherwise I don't think it would run) your code outputs fine for me :
2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53 59 61 67 71 73 79 83 89 97 101 103 107 109 113 127 131 137 139 149 151 157 163 167 173 179 181 191 193 197 199 211 223 227 229 233 239 241 251 257 263 269 271 277 281 283 293 307 311 313 317 331 337 347 349 353 359 367 373 379 383 389 397 401 409 419 421 431 433 439 443 449 457 461 463 467 479 487 491 499 503 509 521 523 541 547 557 563 569 571 577 587 593 599 601 607 613 617 619 631 641 643 647 653 659 661 673 677 683 691 701 709 719 727 733 739 743 751 757 761 769 773 787 797 809 811 821 823 827 829 839 853 857 859 863 877 881 883 887 907 911 919 929 937 941 947 953 967 971 977 983 991 997 1009 1013 1019 1021 1031 1033 1039 1049 1051 1061 1063 1069 1087 1091 1093 1097 1103 1109 1117 1123 1129 1151 1153 1163 1171 1181 1187 1193 1201 1213 1217 1223 1229 1231 1237 1249 1259 1277 1279 1283 1289 1291 1297 1301 1303 1307 1319 1321 1327 1361 1367 1373 1381 1399 1409 1423 1427 1429 1433 1439 1447 1451 1453 1459 1471 1481 1483 1487 1489 1493 1499 1511 1523 1531 1543 1549 1553 1559 1567 1571 1579 1583 1597 1601 1607 1609 1613 1619 1621 1627 1637 1657 1663 1667 1669 1693 1697 1699 1709 1721 1723 1733 1741 1747 1753 1759 1777 1783 1787 1789 1801 1811 1823 1831 1847 1861 1867 1871 1873 1877 1879 1889 1901 1907 1913 1931 1933 1949 1951 1973 1979 1987 1993 1997 1999 2003 2011 2017 2027 2029 2039 2053 2063 2069 2081 2083 2087 2089 2099 2111 2113 2129 2131 2137 2141 2143 2153 2161 2179 2203 2207 2213 2221 2237 2239 2243 2251 2267 2269 2273 2281 2287 2293 2297 2309 2311 2333 2339 2341 2347 2351 2357 2371 2377 2381 2383 2389 2393 2399 2411 2417 2423 2437 2441 2447 2459 2467 2473 2477 2503 2521 2531 2539 2543 2549 2551 2557 2579 2591 2593 2609 2617 2621 2633 2647 2657 2659 2663 2671 2677 2683 2687 2689 2693 2699 2707 2711 2713 2719 2729 2731 2741 2749 2753 2767 2777 2789 2791 2797 2801 2803 2819 2833 2837 2843 2851 2857 2861 2879 2887 2897 2903 2909 2917 2927 2939 2953 2957 2963 2969 2971 2999 3001 3011 3019 3023 3037 3041 3049 3061 3067 3079 3083 3089 3109 3119 3121 3137 3163 3167 3169 3181 3187 3191 3203 3209 3217 3221 3229 3251 3253 3257 3259 3271 3299 3301 3307 3313 3319 3323 3329 3331 3343 3347 3359 3361 3371 3373 3389 3391 3407 3413 3433 3449 3457 3461 3463 3467 3469 3491 3499 3511 3517 3527 3529 3533 3539 3541 3547 3557 3559 3571 3581 3583 3593 3607 3613 3617 3623 3631 3637 3643 3659 3671 3673 3677 3691 3697 3701 3709 3719 3727 3733 3739 3761 3767 3769 3779 3793 3797 3803 3821 3823 3833 3847 3851 3853 3863 3877 3881 3889 3907 3911 3917 3919 3923 3929 3931 3943 3947 3967 3989 4001 4003 4007 4013 4019 4021 4027 4049 4051 4057 4073 4079 4091 4093 4099 4111 4127 4129 4133 4139 4153 4157 4159 4177 4201 4211 4217 4219 4229 4231 4241 4243 4253 4259 4261 4271 4273 4283 4289 4297 4327 4337 4339 4349 4357 4363 4373 4391 4397 4409 4421 4423 4441 4447 4451 4457 4463 4481 4483 4493 4507 4513 4517 4519 4523 4547 4549 4561 4567 4583 4591 4597 4603 4621 4637 4639 4643 4649 4651 4657 4663 4673 4679 4691 4703 4721 4723 4729 4733 4751 4759 4783 4787 4789 4793 4799 4801 4813 4817 4831 4861 4871 4877 4889 4903 4909 4919 4931 4933 4937 4943 4951 4957 4967 4969 4973 4987 4993 4999 5003 5009 5011 5021 5023 5039 5051 5059 5077 5081 5087 5099 5101 5107 5113 5119 5147 5153 5167 5171 5179 5189 5197 5209 5227 5231 5233 5237 5261 5273 5279 5281 5297 5303 5309 5323 5333 5347 5351 5381 5387 5393 5399 5407 5413 5417 5419 5431 5437 5441 5443 5449 5471 5477 5479 5483 5501 5503 5507 5519 5521 5527 5531 5557 5563 5569 5573 5581 5591 5623 5639 5641 5647 5651 5653 5657 5659 5669 5683 5689 5693 5701 5711 5717 5737 5741 5743 5749 5779 5783 5791 5801 5807 5813 5821 5827 5839 5843 5849 5851 5857 5861 5867 5869 5879 5881 5897 5903 5923 5927 5939 5953 5981 5987 6007 6011 6029 6037 6043 6047 6053 6067 6073 6079 6089 6091 6101 6113 6121 6131 6133 6143 6151 6163 6173 6197 6199 6203 6211 6217 6221 6229 6247 6257 6263 6269 6271 6277 6287 6299 6301 6311 6317 6323 6329 6337 6343 6353 6359 6361 6367 6373 6379 6389 6397 6421 6427 6449 6451 6469 6473 6481 6491 6521 6529 6547 6551 6553 6563 6569 6571 6577 6581 6599 6607 6619 6637 6653 6659 6661 6673 6679 6689 6691 6701 6703 6709 6719 6733 6737 6761 6763 6779 6781 6791 6793 6803 6823 6827 6829 6833 6841 6857 6863 6869 6871 6883 6899 6907 6911 6917 6947 6949 6959 6961 6967 6971 6977 6983 6991 6997 7001 7013 7019 7027 7039 7043 7057 7069 7079 7103 7109 7121 7127 7129 7151 7159 7177 7187 7193 7207 7211 7213 7219 7229 7237 7243 7247 7253 7283 7297 7307 7309 7321 7331 7333 7349 7351 7369 7393 7411 7417 7433 7451 7457 7459 7477 7481 7487 7489 7499 7507 7517 7523 7529 7537 7541 7547 7549 7559 7561 7573 7577 7583 7589 7591 7603 7607 7621 7639 7643 7649 7669 7673 7681 7687 7691 7699 7703 7717 7723 7727 7741 7753 7757 7759 7789 7793 7817 7823 7829 7841 7853 7867 7873 7877 7879 7883 7901 7907 7919 7927 7933 7937 7949 7951 7963 7993 8009 8011 8017 8039 8053 8059 8069 8081 8087 8089 8093 8101 8111 8117 8123 8147 8161 8167 8171 8179 8191 8209 8219 8221 8231 8233 8237 8243 8263 8269 8273 8287 8291 8293 8297 8311 8317 8329 8353 8363 8369 8377 8387 8389 8419 8423 8429 8431 8443 8447 8461 8467 8501 8513 8521 8527 8537 8539 8543 8563 8573 8581 8597 8599 8609 8623 8627 8629 8641 8647 8663 8669 8677 8681 8689 8693 8699 8707 8713 8719 8731 8737 8741 8747 8753 8761 8779 8783 8803 8807 8819 8821 8831 8837 8839 8849 8861 8863 8867 8887 8893 8923 8929 8933 8941 8951 8963 8969 8971 8999 9001 9007 9011 9013 9029 9041 9043 9049 9059 9067 9091 9103 9109 9127 9133 9137 9151 9157 9161 9173 9181 9187 9199 9203 9209 9221 9227 9239 9241 9257 9277 9281 9283 9293 9311 9319 9323 9337 9341 9343 9349 9371 9377 9391 9397 9403 9413 9419 9421 9431 9433 9437 9439 9461 9463 9467 9473 9479 9491 9497 9511 9521 9533 9539 9547 9551 9587 9601 9613 9619 9623 9629 9631 9643 9649 9661 9677 9679 9689 9697 9719 9721 9733 9739 9743 9749 9767 9769 9781 9787 9791 9803 9811 9817 9829 9833 9839 9851 9857 9859 9871 9883 9887 9901 9907 9923 9929 9931 9941 9949 9967 9973
EDIT the version of indentation I ran:
output = open("output.dat", 'w')
for i in range(2, 10000):
prime = 1
for j in range(2, i-1):
if i%j == 0:
prime = 0
j = i-1
if prime == 1:
output.write(str(i) + " " )
output.close()
print "writing finished"

Your second for should be nested in the first for.
Also, this looks like a homework question. It is not clear how your output is garbage - does it not compute what you want? Or is the output scrambled? Post a copy of the output so we can see!

Don't you want your loops to be nested?
output = open("output.dat", 'w')
for i in range(2, 10000):
prime = 1
for j in range(2, i-1):
if i%j == 0:
prime = 0
j = i-1
if prime == 1:
output.write(str(i) + " " )
output.close()
print "writing finished"

so, you set prime to 1, 9998 times
then you use the final value of i (10000?, 10001?) as an end value
....
to summarize, you have serious indention problems....