Develop Reference

only print the solution in docplex solve()

Using from docplex.cp.model import CpoModel I have written a docplex code.
Model defenition is as follows.
mdl = CpoModel(name="HouseBuilding")
But by the solve() function it is printing unnecessary output with the solution at last.
msol = mdl.solve(TimeLimit=10)
I believe it is printing below things
.
solve status,
solver parameters,
solver information
output log
Sample output as follows.How should I avoid print these information but only the solution.
! -------------------------------------------------- CP Optimizer 12.10.0.0 --
! Maximization problem - 153 variables, 123 constraints
! TimeLimit = 10
! Initial process time : 0.00s (0.00s extraction + 0.00s propagation)
! . Log search space : 330.1 (before), 330.1 (after)
! . Memory usage : 926.0 kB (before), 926.0 kB (after)
! Using parallel search with 4 workers.
! ----------------------------------------------------------------------------
! Best Branches Non-fixed W Branch decision
0 153 -
+ New bound is 385
! Using iterative diving.
! Using temporal relaxation.
0 153 1 -
+ New bound is 372
* 309 155 0.12s 1 (gap is 20.39%)
* 313 387 0.12s 1 (gap is 18.85%)
* 315 552 0.12s 1 (gap is 18.10%)
315 1000 2 1 F !presenceOf(H4-facade(Jack))
* 340 1480 0.12s 1 (gap is 9.41%)
340 2000 2 1 230 = startOf(H3-garden(Jim))
* 346 2343 0.12s 1 (gap is 7.51%)

You only need set the log_output parameter to delete this unnecessary output
msol = mdl.solve(TimeLimit=10, log_output=False)
To print objetive value (aka solution):
print(msol.objective_value)
Finally, if you need to access the solution of your variables, you must iterare your Narray variable and use:
msol[var_name[(i, j, ... , etc. )]]))
I hope this answer will be helpful to you and sorry for my English.

Gensim Summarizer throws MemoryError, Any Solution?

I am trying to generate the summary of a large text file using Gensim Summarizer.
I am getting memory error. Have been facing this issue since sometime, any help
would be really appreciated. feel free to ask for more details.
from gensim.summarization.summarizer import summarize
file_read =open("xxxxx.txt",'r')
Content= file_read.read()
def Summary_gen(content):
print(len(Content))
summary_r=summarize(Content,ratio=0.02)
print(summary_r)
Summary_gen(Content)
The length of the document is:
365042
Error messsage:
---------------------------------------------------------------------------
MemoryError Traceback (most recent call last)
<ipython-input-6-a91bd71076d1> in <module>()
10
11
---> 12 Summary_gen(Content)
<ipython-input-6-a91bd71076d1> in Summary_gen(content)
6 def Summary_gen(content):
7 print(len(Content))
----> 8 summary_r=summarize(Content,ratio=0.02)
9 print(summary_r)
10
c:\python3.6\lib\site-packages\gensim\summarization\summarizer.py in summarize(text, ratio, word_count, split)
428 corpus = _build_corpus(sentences)
429
--> 430 most_important_docs = summarize_corpus(corpus, ratio=ratio if word_count is None else 1)
431
432 # If couldn't get important docs, the algorithm ends.
c:\python3.6\lib\site-packages\gensim\summarization\summarizer.py in summarize_corpus(corpus, ratio)
367 return []
368
--> 369 pagerank_scores = _pagerank(graph)
370
371 hashable_corpus.sort(key=lambda doc: pagerank_scores.get(doc, 0), reverse=True)
c:\python3.6\lib\site-packages\gensim\summarization\pagerank_weighted.py in pagerank_weighted(graph, damping)
57
58 """
---> 59 adjacency_matrix = build_adjacency_matrix(graph)
60 probability_matrix = build_probability_matrix(graph)
61
c:\python3.6\lib\site-packages\gensim\summarization\pagerank_weighted.py in build_adjacency_matrix(graph)
92 neighbors_sum = sum(graph.edge_weight((current_node, neighbor)) for neighbor in graph.neighbors(current_node))
93 for j in xrange(length):
---> 94 edge_weight = float(graph.edge_weight((current_node, nodes[j])))
95 if i != j and edge_weight != 0.0:
96 row.append(i)
c:\python3.6\lib\site-packages\gensim\summarization\graph.py in edge_weight(self, edge)
255
256 """
--> 257 return self.get_edge_properties(edge).setdefault(self.WEIGHT_ATTRIBUTE_NAME, self.DEFAULT_WEIGHT)
258
259 def neighbors(self, node):
c:\python3.6\lib\site-packages\gensim\summarization\graph.py in get_edge_properties(self, edge)
404
405 """
--> 406 return self.edge_properties.setdefault(edge, {})
407
408 def add_edge_attributes(self, edge, attrs):
MemoryError:
I have tried looking up for this error on the internet, but, couldn't find a workable solution to this.

From the logs, it looks like the code builds an adjacency matrix
---> 59 adjacency_matrix = build_adjacency_matrix(graph)
This probably tries to create a huge adjacency matrix with your 365042 documents which cannot fit in your memory(i.e., RAM).
You could try:
Reducing the document size to fewer files (maybe start with 10000)
and check if it works
Try running it on a system with more RAM

Did you try to use word_count argument instead of ratio?
If the above still doesn't solve the problem, then that's because of gensim's implementation limitations. The only way to use gensim if you still OOM errors is to split documents. That also will speed up your solution (and if the document is really big, it shouldn't be a problem anyway).
What's the problem with summarize:
gensim's summarizer uses TextRank by default, an algorithm that uses PageRank. In gensim it is unfortunately implemented using a Python list of PageRank graph nodes, so it may fail if your graph is too big.
BTW is the document length measured in words, or characters?

How to unpickle a python object in Golang

I have a python program wherein I am using Pickle to store the object using the following:
pickle.dump(sample, open( "Pickled_files/sample.p", "wb" ))
I can extract and unpickle this object in Python using the following:
sample_extracted= pickle.load(open( "Pickled_files/sample.p", "rb" ))
However, I need to extract this object in a Golang application. Thus I need to know a way by which objects pickled using Python are extracted in Golang.
Is there a way that this can be achieved? And if yes, I would really appreciate if someone can point me to a sample reference or example.

ogórek (2) is Go library for decoding/encoding Python pickles.
A pickle saved to the file can be loaded in Go as follows:
f, err := os.Open("Pickled_files/sample.p")
d := ogórek.NewDecoder(f)
obj, err := d.Decode()

Pickle is Python specific format. AFAIK there are no pickle-parsers outside of Python. You can try to write one for Go but you will most likely only waste lots of time and mental health. On the other hand that would be an interesting project, indeed.
Anyway, instead of pickling use any language independent format, i.e. xml, json, google's protobuf or even a custom one, whatever suits your needs. Always pick tool for a job, never other way around.

Is there a way that this can be achieved?
Depends on your understanding of this. There are a lot of better options than Pickle -- even in pure Python environments. If you understand this as exchanging data between golang and Python, you should consider the following example:
You can serialize everything in Python, like
import msgpack
import msgpack_numpy as m
m.patch()
import numpy as np
data = {'string': 'Hello World',
'number': 42,
'matrix': np.random.randn(2, 3)}
with open('data.msgp', 'wb') as f:
f.write(msgpack.packb(data, use_bin_type=True))
Reading it is pretty simple
// go get -u github.com/vmihailenco/msgpack
package main
import (
"fmt"
"github.com/vmihailenco/msgpack"
"io/ioutil"
)
func main() {
buf, err := ioutil.ReadFile("data.msgp")
if err != nil {
panic(err)
}
var out map[string]interface{}
err = msgpack.Unmarshal(buf, &out)
if err != nil {
panic(err)
}
for k, v := range out {
fmt.Printf("key[%v] value[%v]\n", k, v)
}
}
This gives you
key[matrix] value[map[data:[145 106 174 12 61 187 235 63 128 225 138 214 167 154 231 191 156 205 144 51 50 116 244 191 251 147 235 33 187 149 251 63 207 56 134 174 206 146 220 63 7 23 246 148 34 235 226 63] type:[60 102 56]
kind:[] nd:true shape:[2 3]]]
key[string] value[[72 101 108 108 111 32 87 111 114 108 100]]
key[number] value[42]
All is left is converting the byte sequences into the object you would like to have.

graphviz_layout: AttributeError: 'NoneType' object has no attribute 'get_node'

For some reason when i try to implement this function in my dynamically created graph, I get this weird error. I can run the function examples given online but its fails when I run it.
stacktrace:
122 #pos = nx.spectral_layout(G)
123 #write_dot(G,'test.dot')
--> 124 pos= graphviz_layout(G,prog='twopi',args='')
125 nx.draw_networkx(G, node_color=nodeColors, arrows=False, alpha=.8, labels=nodeLabels, font_size=8)
126 print(nx.info(G))
246 This is a wrapper for pydot_layout.
247 """
--> 248 return pydot_layout(G=G,prog=prog,root=root,**kwds)
249
250
281 for n in G.nodes():
282 pydot_node = pydotplus.Node(make_str(n)).get_name()
--> 283 node=Q.get_node(pydot_node)
284
285 if isinstance(node,list):
AttributeError: 'NoneType' object has no attribute 'get_node'

It looks to me like this is a simple typo. You wrote Q where you probably meant G. There is nothing called Q, and so Q.get_node makes no sense. However G.get_node would make sense.

Without understanding more about the code, it is hard to say what caused the problem, but...
Based on the error message and the stacktrace, the object 'Q' is set to None. For some reason, when the object Q was made, the code assigned the value None instead of the desired value. Troubleshooting will require backtracking to find out when Q was made and why it was set to None.
--> 283 node=Q.get_node(pydot_node)

The answer from Joel may not be correct, since from the source, it shows that
P=to_pydot(G)
if root is not None :
P.set("root",make_str(root))
D=P.create_dot(prog=prog)
if D=="": # no data returned
print("Graphviz layout with %s failed"%(prog))
print()
print("To debug what happened try:")
print("P=pydot_from_networkx(G)")
print("P.write_dot(\"file.dot\")")
print("And then run %s on file.dot"%(prog))
return
Q=pydotplus.graph_from_dot_data(D)
node_pos={}
for n in G.nodes():
pydot_node = pydotplus.Node(make_str(n)).get_name()
node=Q.get_node(pydot_node)
So Q plays no direct role with the eternal

Mysterious IndexError

I've been getting errors similar to this recently:
IndexError Traceback (most recent call last)
<ipython-input-124-59ca523b1b36> in <module>()
----> 1 first_experiment_comb(model)
c:\python26\26664\lib\site-packages\experiments.py in first_experiment_comb(mod
l)
172 "Number NZ: " + str(modelz[j].NumNZs) +"\n")
173
--> 174 first_experiment(modelz[j], str(j))
175
176
c:\python26\26664\lib\site-packages\experiments.py in first_experiment(model, e
t)
89 plt.close()
90
---> 91 fl.timberFlow(model)
92 plt.savefig(dire + "\\timber_flow" +ext+".pdf", bbox_inches = 0)
93 plt.close()
C:\Python26\26664\lib\site-packages\func_lib.py in timberFlow(model)
304 if not unVars:
305 unVars = varValues(model, 'PIEHTLVOL')
--> 306
307 for i in range(19):
308 swVarVals.append(swVars[i].X)
IndexError: list index out of range
Where the final line of the trace points to code that doesn't exist, or in previous cases has been commented out. When I run the last function (in func_lib.py) on it's own I never get the mysterious IndexError, only when it's called from experiments.py.
I'm running this in pylab python 2.6 W64.
I haven't been able to find known bug in iPython or Pylab docs about this.
How could line 306 be the root of the error?

Your code is out of sync with the bytecode. Reload your code properly.
When an exception occurs, the bytecode is inspected for a filename and a linenumber, and then the sourcefile is loaded to show the original source for that line.
If, however, you changed the source but did not yet restart your python process (or reloaded the code in ipython) then the wrong lines are being shown when an exception occurs.