Iacutone.rb

coding and things

Python, First Impressions

| Comments

Debugging

  • Write the following code to use an interactive debugger in Python.
1
2
import pdb
pdb.set_trace()

Self

  • Passing a reference to self seems repetitive.
1
2
3
class Foo:
  def bar(self)
    print('Hello')

Testing

  • You need to pass in the -s to print test output. I expected print to log output regardless.

Modules

  • You can conditionally implement different behavior when importing modules in Python.
  • Taken from this StackOverflow example.
one.py
1
2
3
4
5
6
7
8
9
def func():
    print("func() in one.py")

print("top-level in one.py")

if __name__ == "__main__":
    print("one.py is being run directly")
else:
    print("one.py is being imported into another module")
two.py
1
2
3
4
5
6
7
8
9
import one

print("top-level in two.py")
one.func()

if __name__ == "__main__":
    print("two.py is being run directly")
else:
    print("two.py is being imported into another module")

Running python one.py returns: top-level in one.py one.py is being run directly

While running python two.py returns: top-level in one.py one.py is being imported into another module top-level in two.py func() in one.py two.py is being run directly

Artificial Intelligence Nano Degree

| Comments

I started the Udacity Artificial Intelligence Nano Degree last month.

Project 1, Sudoku Solver

The Sudoku Solver was a fun project. I enjoy learning by writing code to make tests pass, like Ruby Katas.

Project 2, Isolation Game

Creating an algo to play an Isolation Game is much more complicated than the first project and was very frustrating to build. Unlike the first project, there is barely a test framework to follow. You push your methods to a remote Udacity server which gives feedback about test failures. This development cycle is difficult to debug because you can neither use a debugger nor get print output. After the minimax and alphabeta functions pass the remote tests, things get interesting. You are asked to write custom heuristics in order to decide the best move for your player. I wish Udacity gave more hints as to game heuristics.

Writing a synopsis for a research paper is also required for the project. I chose to write about AlphaGo. Due to my lack of knowledge reading math proofs, it took several attempts to understand the paper.

Project 3, Implement a Planning Search

Using planning problems—states, actions, and goals for planning algorithms.

  • This Computerphile video is a good reinforcement for A* search.

My Photo Storage Solution

| Comments

I have been searching for a photo storage solution. My solution only involves two devices, a raspberry pi and an external hard drive. This post will explain how to set up your pi in order to automatically transfer RAW and JPG photos from an SD card to the hard drive connected to the pi. The scripts I used are available here.

I like this solution because the entire process is automatic. You are notified when the photo transfer has finished with an email. Also, for redundancy, I created a cron job that syncs all of the photos on my hard drive to Amazon photos. I use this service to sync the photos in the cloud.

Steps

Mount the external hard drive to a folder on your pi.
  • sudo fdisk -l take note of the Device, something like /dev/sdb1
  • I like to mount the drive under the home dir, sudo mount /dev/sdb1 ~/external_hd/
  • Run lsblk -f to know here files are mounted
Create a udev rule
  • udev rules live here: /etc/udev/rules.d
  • cd /etc/udev/rules.d
  • touch 50-sdcard.rules
  • chmod +x 50-sdcard.rules
  • sudo fdisk -l again to find the sd card device name, in this instance ‘sda1’
  • echo 'KERNEL=="sda1", ACTION=="add", RUN+="/home/pi/sdcard/sdcard_added.sh"' < 50-sdcard.rules
  • tail -f /var/log/syslog is extremely useful for debugging udev rules
Create an fstab privilege
  • vi /etc/fstab
  • Find device uuid with blkid
  • Add UUID=<your-uuid> /home/pi/SD_CARD auto rw,users,noauto 0 0
  • Uncomment out ‘user_allow_other’ in /etc/fuse.conf
rsync when an sd card is added
  • touch /home/pi/sdcard/sdcard_added.sh
  • chmod +x /home/pi/sdcard/sdcard_added.sh
  • create the script
1
2
3
4
5
#!/bin/bash

echo "`date +%Y-%m-%d:%H:%M:%S` SD card inserted" >> /home/pi/sdcard/output.log
mount /dev/sda1 /home/pi/SD_CARD
rsync -av /home/pi/SD_CARD/DCIM/100MSDCF/ /home/pi/external_hd/raw_photos
Sync photos to cloud provider
  • Set up cron job
  • sudo crontab -e
  • 00 4 * * * /home/pi/sd_card/rclone_cron.sh, this runs the cron job at 4 am, daily
  • create the bast script
1
2
#!/bin/sh
/usr/sbin/rclone sync /home/pi/external_hd/raw_photos/ amazon:raw_photos --config /home/pi/.rclone.conf -v
Send Email Notification
  • I used this post to easily add email notifications when the rsync process completes
Tips Since Creating My Photo Process
  • I could not get the above code to work with Debian due to permission issues
  • It is better to identify from UUID instead of KERNEL for setting up udev rules, run ls -l /dev/disk/by-uuid to discover device UUID’s
RClone
  • Amazon has blocked all thrid party applications from pushing to Amazon Drive. For more information see this
  • I am going to implement Duplicati as a solution to sync my photos to Amazon Drive
  • Another potential solution is to continute to use rclone, but host the files on Google Drive

RSpec and Elastic Search

| Comments

I had a difficult time setting up ElasticSearch on both RSpec (CircleCI) and Heroku. The ElasticSearch test cluster was not working on the CircleCI Docker image. Fortunately, one can configure the Circle environment to start with an ElasticSearch process. So, instead of using the test cluster, both my local testing environment and Circle environment use a real ElasticSearch process.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#
# ElasticSearch
#

config.before :all, elasticsearch: true do
  port = ENV['CIRCLE_CI_ES_URL'].present? ? 9200 : 9250
  Elasticsearch::Model.client = Elasticsearch::Client.new(port: port)
end

config.before :each, elasticsearch: true do
  Campaign.__elasticsearch__.create_index!(force: true)
end

config.after :each, elasticsearch: true do
  Campaign.__elasticsearch__.delete_index!
end

When a test needs to use ElasticSearch:

1
2
3
4
5
before { Campaign.__elasticsearch__.refresh_index! }

describe '#search', elasticsearch: true do
  expect(search.results).to be_present
end

I ran into issues using ElasticSearch on Heroku when creating an index. Heroku review apps are configurable by defining an app.json. In the json file, Heroku can spin up an ElasticSearch process.

1
2
3
4
5
6
7
8
9
10
11
12
13
"scripts": {
  "postdeploy": "bundle exec rake db:schema:load db:seed"
},
"formation": {
  "web": {
    "size": "free",
    "quantity": 1
  },
  "elasticsearch": {
    "size": "free",
    "quantity": 1
  }
}

The process is running before any Ruby code is executed. The next step is to create the index using postdeploy. Before creating ActiveRecord objects in the seed.rb file, create an index with Model.index(force: true).

Vim, Thoughts After Two Weeks

| Comments

Purposefully, I began using Vim during a non-strenuous work week which helped manage frustration. I have been using solely Vim as a text editor for two weeks and have experienced improved efficienies over my previous text editor, Sublime.

Improvments

The ability to change between files quickly using the Ctrl-P plugin is the biggest quality of life increase. Commands such as / and ? make finding word matches in a file easy. Using RSpec with Vim is also great; I can change my code and re-run a single test without switching screens! There are many small wins with using Vim and they add up.

Pain Points

Adding new files does not work if the directory does not exist. The mkdir plugin solves this issue. Furthermore, commenting out code is a pain without vim-commentary. Checking my vimrc file into version controller has been helpful in debgging where breaking changes originate.

More Resources

Vim Awesome

Level Up With Vim

| Comments

How much time do you spend in your text editor? We spend most of our time reading through text and jumping between files. My goal is to find a tool in order to help me do this more efficiently. I have found that learning Vim is not overwhelming or difficult if you break the units of work down into manageable pieces. The ultimate goal is to use Vim fulltime. Below are my curated steps:

Intro Video
Vimtutor
  • Comprehensive introduction to Vim
  • Type vimtutor in a Terminal session
  • Complete a few sections per day
Screencasts
Advanced
Helpful Links

Filtering Ack Results

| Comments

While setting up my VIM environment, I read a blogpost which states ack can ignore directories. For cleaner ack output you can setup the file below:

.ackrc
1
2
3
4
--ignore-dir=log
--ignore-dir=public/assets
--ignore-dir=vendor/assets
--ignore-dir=tmp/cache

Rails `before_action` Method

| Comments

I did not know you could give a before_action a block argument. This is a benefit because you do not have to create a ruby method and pass the method into the before_action.

This got me thinking, what does before_action look like in Rails? The following is the source code from Rails 4.2.7, from AbstractController::Callbacks::ClassMethods.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# :method: before_action
#
# :call-seq: before_action(names, block)
#
# Append a callback before actions. See _insert_callbacks for parameter details.

# Take callback names and an optional callback proc, normalize them,
# then call the block with each callback. This allows us to abstract
# the normalization across several methods that use it.
#
# ==== Parameters
# * <tt>callbacks</tt> - An array of callbacks, with an optional
#   options hash as the last parameter.
# * <tt>block</tt>    - A proc that should be added to the callbacks.
#
# ==== Block Parameters
# * <tt>name</tt>     - The callback to be added
# * <tt>options</tt>  - A hash of options to be used when adding the callback

def _insert_callbacks(callbacks, block = nil)
  options = callbacks.extract_options!
  _normalize_callback_options(options)
  callbacks.push(block) if block
  callbacks.each do |callback|
    yield callback, options
  end
end

[:before, :after, :around].each do |callback|
  define_method "#{callback}_action" do |*names, &blk|
    _insert_callbacks(names, blk) do |name, options|
      set_callback(:process_action, callback, name, options)
    end
  end
end

And I can do cool things like:

1
2
3
before_action -> {
  @foo = Model.find params[:bar]
}

I would like to point out the set_callback method.

1
2
3
4
before_action :authenticate

# can also be called like this
set_callback :process_action, :before, :authenticate

Extend and Include in Ruby

| Comments

I have been trying to clean up some old code with Ruby modules. This post is to help me remember the differences between include and extend in Ruby.

1
2
3
class Foo
  extend ActionView::Helpers::NumberHelper
end
1
2
3
module Foo
  extend ActionView::Helpers::NumberHelper
end
1
2
Foo.number_to_currency 2
 => "$2.00"
1
2
3
class Foo
  include ActionView::Helpers::NumberHelper
end
1
2
3
foo = Foo.new
foo.number_to_currency 2
 => "$2.00"

Find and Replace Text

| Comments

If you find yourself in the situation of needing to find and replace text in multiple files, use the Command Line Interface. I wanted to rename a Phoenix application and ran the below command.

ack pivotal_commentor -l | xargs sed -i '' 's/pivotal_commentor/commentor/g'

Let’s break this command down with a simple example.

1
2
3
4
mkdir test
cd test
touch file1.txt
touch file2.txt

And lets add the text ‘hello’ to both text files.

1
2
3
ack hello -l
file1.txt
file2.txt

ack is like grep and found the files containing the string hello. The -l flag “Only print filenames containing matches”

1
2
ack hello -l | xargs
file1.txt file2.txt

xargs is a Unix utility that constructs argument lists and is smashing the filenames into one line.

Finally, we pipe the arguments from xargs to sed.

1
ack hello -l | xargs sed -i '' 's/hello/bye/g'

The -i flag allows for in place editing on the file. In the regex, the s replaces hello with bye and the g indicates globally, in case hello is found more than once in the file. The ‘’ is sending the change into the correct output file.