Ceci est une ancienne révision du document !
*Lately, I've been doing some work with a computer book publisher on Machine Learning and Python. It's a very interesting subject, and I really enjoy the learning process on all of the various modeling methods. You want pickle with that? One thing that I've found is that when you need to (or want to) save some of the data in the middle of a process, often the pickle library is used. I've known about pickle for a long time, but have really never messed with it much, so I thought I'd explore some.
Pickle or olive? An olive is a wonderful thing in a martini. It doesn't do anything for Python code. It is, however, a type of non-venomous Python mainly in Australia. Pickle on the other hand, is a method to serialize and deserialize Python object structures. If you know all there is to know about Pickles, feel free to jump forward in the article. If not, or if you trust me to teach you something, keep reading.
Serializing means to take an object from memory, convert it into a stream of bytes that can be stored on disk. De-serializing is the reverse of the process. Let's say you have a dictionary. You can't just dump it to disk from memory. You have to convert it into a format that lends itself to being a disk file. JSON, XML, HTML all jump to mind. Pickles are just another way to do this. There's a library that handles all the tough stuff for you. What can you pickle? Well most Python objects can be pickled, but there are a few that can't. While you can pickle simple objects (Integers, floats, complex numbers and strings), you normally would pickle Tuples, Lists, sets and Dictionaries that are built from most objects. However, things like generators, lambda functions and defaultdicts can not be pickled. There are some workarounds, but this is pretty much the rule of thumb.
Pickling Process I want to thank my friend Halvard Tislavoll from Norway for the idea and the code for this part. Let's say that you want to create a dataset of colors that can be used when creating a GUI. Tkinter (as well as other GUI toolkits) allows you to use hex codes as well as color names. There are many web pages that show all the colors along with the color name for quick reference, but what if you needed all of them along with their hex values? Under Ubuntu, there is a file located in the /etc/X11 folder called rgb.txt. Make a copy of it and put the copy into a working folder.
Please note that Halvard's coding style is not the same as mine and I'm sure that it's not the same as yours. However, I'm sure that you will be able to understand his code. Now we can get started creating a program to convert this text to a dictionary and pickle it. Name your program file “rgb2pickle.py”. First, you need to import the pickle library import pickle Now, let's define an empty list and empty dictionary… myList = [] myDict = {}
Next, we'll want to create some support functions. First the function that will read the rgb.txt file and return the contents in a list (see next page, top right). This function takes the decimal value and returns the hex value using the above helper functions (next page, bottom right). This function takes the list and puts everything into a dictionary… def txt2dict(myList): i = 0 aDict = {} for item in myList: newList = [] # pick name, clean it for tab name = item[13:].lstrip(“\t”) name = name.rstrip() # pick RGB values (dec) R = item[0:3].lstrip() R = emptystring(R) G = item[4:7].lstrip() G = emptystring(G) B = item[8:11].lstrip() B = emptystring(B)
Now, convert the RGB values to a hex value… # convert dec2hex value H1 = dec2hex(R) H2 = dec2hex(G) H3 = dec2hex(B) # build a hash value H = f“#{H1}{H2}{H3}” And append this to the list structure and update the dictionary… # make a list with a new column structure newList.append(name) newList.append(R) newList.append(G) newList.append(B) newList.append(H) aDict[i] = newList i += 1 return aDict
Now that all of the helper functions are done, let's put them all together (next page, top right). The terminal output is very uninteresting and responds very quickly. Now, we have a pickle file, what do we do with it? Depickling a pickle Depickling (or deserializing) is just as easy as it was to create the pickle file once you have the data ready. Here is a quick code snippet that you can use in a CLI application… with open(filename, 'rb') as f: data = pkl.load(f)
This works well only if the pickle file was created with Python 3. If, however, it was created with Python 2 and the cPickle routine, it will probably error out. An easy workaround for this is (shown bottom right). Anyway, for my purposes I wanted to be able to actually see the data in its raw form, direct from the pickle file. I threw together a quick Page form and threw in a very little code (THANK YOU Page!) and here is the result… I won’t bore you with the details of how to do this in Page, since there’s only three buttons, an entry widget, two labels, and a scrolled text widget. We’ve already covered that. However, I will show you the code that is important in the _support file. I really didn’t worry about any error checking for the simple project.
We’ll look at the callback for the “get filename” button (the one that has “…” as its text) first (middle right). Basically, this simply calls a tkinter askopenfilename filedialog and puts the selected filename and path into the entry widget for display. Next is the callback for the “GO” button. This is where the real work is done. The logic is to: • Clear the text box. • Open the file that the user has selected in the routine above. • Depickle it. • Determine the type of data structure and display it. • Fill the text widget with the data from the structure (if possible).
def on_btnGo(): # print('depickle1_support.on_btnGo') # sys.stdout.flush() clear_stw() # Clear the text widget Here is the actual code for depickling (next page, top right). Lastly, based on the type of data we have, fill the text widget (bottom right). The function to clear the text widget is really simple… def clear_stw(): # Clear the ScrolledText Widget w.Scrolledtext1.delete('1.0', 'end')
And the very last thing, just to be complete, is the Page provided function that defines the tkinter variables that allow easy setting of the text for the various widgets. def set_Tk_var(): global pickleType pickleType = tk.StringVar() pickleType.set('') global entry_var entry_var = tk.StringVar() That’s it.
Here is a good website that can help you understand the pickling process if you still want to learn more… https://www.datacamp.com/community/tutorials/pickle-python-tutorial I’ve put the source files for the rgb2pickle.py file from Halvard on pastebin at: https://pastebin.com/s09mp72G And the python source for the Depickle GUI program there as well at: Depickle.py - https://pastebin.com/wsUMqk1F; Depickle_support.py - https://pastebin.com/TgmgngxJ Until next time, I hope you have a wonderful New Year and remember to keep coding!