def(x: Experience):Code

I recently developed a Jupyter notebook and needed to convert it to a Python script quickly. Unfortunately, the File -> Download as -> Python (.py) option didn't work.  This script will open a Jupyter notebook and convert it to a script:

Recently, I've been looking at a problem where data is joined with enrichment data stored in a Pandas dataframe. For each record being processed, a lookup is performed and the single record from the enrichment data selected based on a key. Essentially, this is a SQL join operation, but one of the datasets couldn't fit into memory. Having had experience with dataframes being slow (particularly when iterating through rows), I investigated whether the lookup would be faster if a enrichment data was stored in a Python dict as opposed to a dataframe. In my experiments, I was able to get a 70 times speed improvement using a dict over a dataframe, even when indexing the dataframe. The Python code used in the experiments is here:

Getting Neo4j 4.1.0 to work in Docker has been a real struggle! The docker-compose file was: version: "3" services: neo4j: image: neo4j container_name: neo4j ports: - 7474:7474 - 7687:7687 environment: - "NEO4J_AUTH=none" Note that authentication has been turned off, so just login with a blank username and password. The browser kept returning: WebSocket connection failure. Due to security constraints in your web browser, the reason for the failure is not available to this Neo4j Driver. Please use your browsers development console to determine the root cause of the failure. Common reasons include the database being unavailable, using the wrong connection URL or temporary network problems. If you have enabled encryption, ensure your browser is configured to trust the certificate Neo4j is configured to use. WebSocket `readyState` is: 3 To solve this, when you login change neo4j:// to bolt://

The Problem Development and test clusters may be air-gapped so that client data or sensitive software under development is less likely to be leaked. This can cause problems when trying to install libraries, e.g. for Python-based software, especially if the cluster has an old version of a Linux OS installed. The Solution Setup Python's pip on the remote machine On an Internet-enabled machine, download the Wheel file for pip from  https://pypi.python.org/pypi/pip , such as pip-9.0.1-py2.py3-none-any.whl. Copy the Wheel file (e.g. pip-9.0.1-py2.py3-none-any.whl) to the remote machine, e.g. using scp: scp pip-9.0.1-py2.py3-none-any.whl user@host:/path On the remote machine: python pip-9.0.1-py2.py3-none-any.whl/pip install --no-index pip-9.0.1-py2.py3-none-any.whl pip --version # this should display the version number if correctly installed Download the required libraries On an Internet-enabled machine, download the library and its d...

This provides a quick introduction to setting up a local Kafka instance and using Kafka Tool to view the messages. The initial part of the Kafka tutorial is adapted slightly from  http://kafka.apache.org/quickstart.html . The following are steps for: Downloading and unpacking Apache Kafka in Linux Starting the Zookeeper and Kafka servers Creating a new topic Adding and viewing messages using the command line tools Use Kafka Tool to view the messages in a topic Step 1: Download Kafka cd ./Downloads/ wget http://mirror.catn.com/pub/apache/kafka/0.10.0.0/kafka_2.11-0.10.0.0.tgz tar -zxf kafka_2.11-0.10.0.0.tgz cd kafka_2.11-0.10.0.0 Step 2: Start the Zookeeper and Kafka servers Check the Kafka port in zookeeper.properties by looking at clientPort (typically 2181) in config/zookeeper.properties Start Zookeeper with: bin/zookeeper-server-start.sh config/zookeeper.properties and then start Kafka: bin/kafka-server-start.sh config/server.properties ...

Generating nullable objects in Scala ScalaCheck has completely changed how I approach testing. In a recent problem, I needed to be able to easily generate objects that could be null. Here's how I approached it: -

Apache Pig is fantastic for writing Big Data algorithms without having to write Map-Reduce jobs from scratch. I recently encountered a problem where I needed to get the maximum and minimum values in a column of data. For example, say the data (in CSV format) looks like this: Cat, 5 Mouse, 8 Dog, 4 This Pig Latin script reads in the data and finds the required values: animal_ages = LOAD 'data.csv' USING org.apache.pig.piggybank.storage.CSVExcelStorage() AS (animal:chararray, age:int); ages = FOREACH animal_ages GENERATE age; ages_grp = GROUP ages ALL; min_age = FOREACH ages_grp GENERATE MIN(ages) as min_val; max_age =  FOREACH ages_grp GENERATE MAX(ages) as max_val; The minimum and maximum ages can be used as a scalar: min_age.min_val max_age.max_val

I downloaded RealOne Player, but the installation failed because it said that my version of ActiveSync was incorrect. In order to get it working follow these steps: Start ActiveSync Double-click the RealPlayer installation file Get the error, but don't click 'Ok' on the error message Type %temp% in Run, i.e. Start->Run->Type '%temp' Open the folder named ~rnsetup Copy the realplayer.cab file in that folder onto the PDA In 'File Explorer' on the PDA double-click the .cab file to install. These steps were suggested on this forum . For a bit more information about installing .cab files read: http://pocketpccentral.net/help/tutorials/install_cab_file.htm

On the PDA: Click 'Start' -> 'Settings' -> 'Connections' -> 'Bluetooth' On the PDA: Ensure Bluetooth is turned on In BlueSoleil:Click 'My Bluetooth' -> 'Security' -> Set to High, Set a Default Passkey and click 'Enable Data Encryption' In BlueSoleil: Right-click on the PDA -> Click 'Pair' On the PDA: You will be asked to enter the pin code to pair with the computer. Enter the Default Passkey you typed and click Ok In BlueSoleil: Right-click on the PDA -> Click 'Connect' --> Click 'Bluetooth Serial Port Service' In BlueSoleil: Right-click on the PDA -> Click 'Status...' --> Note the COM port number On the PDA: Click 'Start' -> 'Settings' -> 'Connections' -> 'Bluetooth' --> 'Bluetooth Manager' On the PDA: Click 'New' -> 'ActiveSync via Bluetooth' -> Select the computer -> Click a serial port -...

Introduction Having successfully installed Python on my Windows Mobile PDA I wanted to investigate if I could communicate between my Windows XP machine and my PDA using sockets. I used Cygwin to act as a Linux emulator, although there are Windows-based Python solutions, such as Enthought's Pylab . Step One: Communication between processes on the same machine As a first step, try communication between processes on a single machine. I used the command ipconfig in a Windows command prompt to determine that my IP address is 192.168.1.64. To create the server, I used the following Python code, which is largely from http://www.devshed.com/c/a/Python/Sockets-in-Python/1/ . Create a file called server.py with the following contents: #! /usr/bin/env python import socket mySocket = socket.socket (socket.AF_INET, socket.SOCK_STREAM) mySocket.bind (( '192.168.1.64', 1200 )) mySocket.listen (1) while True: channel, details = mySocket.accept() print 'Opened a connection with...

I've been doing development work in Python and as an experiment I thought I'd have a go at getting Python installed on my Windows Mobile PDA . It's an HP iPaq running Windows Mobile 6 (CE OS 5.2.1.1616). The Python CE Wiki is located at: http://pythonce.sourceforge.net/Wikka/HomePage I installed PythonCE-2.5-20061219-setup.exe from sourceforget.net and amazingly it worked first time! After clicking Start -> Programs -> Python I was able to verify that it worked by typing >>> print 'Hello World' Hello World Using Ilium Software Screen Capture software I was able to get a screen shot very easily. It can be downloaded from http://www.mobiletopsoft.com/pocket-pc/download-ilium-software-screen-capture-free-1-1.html