Iacutone.rb

coding and things

Understanding Test Doubles

| Comments

I refactored credit card payments on an application that uses the ActiveMerchant gem. I was not confident in the test, so, I re-wrote the tests to verify responses from Braintree. Once I verified I did not break anything with my refactor, I decided the best next step was to remove the interaction with Braintree and replace it with stubbed responses. I always forget when and how to use RSpec allow, spy and double methods. This post is meant to help reinforce my knowledge of the aforementioned methods.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
describe CreditCardPayment do
  let!(:payment) { build(:credit_card_payment) }

  describe '#authorize' do
    before { payment.authorize }

    it 'records the GatewayTransaction' do
      transaction = GatewayTransaction.last

      expect(transaction.success).to eq true
      expect(transaction.amount).to eq amount
      expect(transaction.message).to eq '1000 Approved'
      expect(transaction.response).to be_present
      expect(transaction.service_fee).to eq service_fee[:service_fee]
    end
  end
end

Authorize is a wrapper around the payment gateway #authorize method which returns a response from Braintree. I should have confidence that when sending Braintree arguments that the#authorize expects, the response will be successful. This is interaction is what needs to be stubbed in order to not call the Braintree API in the test environment. The benefits of this approach are that the test will be faster and more resilient because there is no communication with an external service. The negatives of this approach is that the API can change.

Let’s update the above codeblock to stub out the interaction with Braintree.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
describe CreditCardPayment do
  let(:gateway_double) { double('ActiveMerchant::Billing::BraintreeGateway') }
  let!(:payment) { build(:credit_card_payment, gateway: gateway_double) }

  before { payment.gateway = gateway_double }

  describe '#authorize' do
    it 'sends a #authorize request to Braintree' do
      expect(gateway_double).to receive(:authorize).exactly(1).times
        .with(amount, payment.credit_card, payment.credit_card_descriptor)
  .and_return(authorize_response_stub)

      payment.authorize
    end

    it 'creates a GatewayTransaction' do
      expect { payment.authorize }.to change(GatewayTransaction, :count).by(1)
    end
  end

  def authorize_response_stub
    params = { 'response_from_braintree' => 'yay' }

     ActiveMerchant::Billing::Response.new(true, '1000 Approved', params, {authorization: '3e4r5q'})
  end
end

Notice that in the it block the expectation comes before the #payment method call. We could replace the double with a spy. The difference is a spy uses the have_received method. It is your preference to use either a double or a spy.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
  let(:gateway_double) { spy('ActiveMerchant::Billing::BraintreeGateway') }
  let!(:payment) { build(:credit_card_payment, gateway: gateway_double) }

  before { payment.gateway = gateway_double }

  describe '#authorize' do
    it 'sends a #authorize request to Braintree' do
      payment.authorize

      expect(gateway_double).to have_receive(:authorize).exactly(1).times
        .with(amount, payment.credit_card, payment.credit_card_descriptor)
  .and_return(authorize_response_stub)
    end

    it 'creates a GatewayTransaction' do
      expect { payment.authorize }.to change(GatewayTransaction, :count).by(1)
    end
  end

Helpful Links

RSpec Mocks 3.6

Python, First Impressions

| Comments

Debugging

  • Write the following code to use an interactive debugger in Python.
1
2
import pdb
pdb.set_trace()

Self

  • Passing a reference to self seems repetitive.
1
2
3
class Foo:
  def bar(self)
    print('Hello')

Testing

  • You need to pass in the -s to print test output. I expected print to log output regardless.

Modules

  • You can conditionally implement different behavior when importing modules in Python.
  • Taken from this StackOverflow example.
one.py
1
2
3
4
5
6
7
8
9
def func():
    print("func() in one.py")

print("top-level in one.py")

if __name__ == "__main__":
    print("one.py is being run directly")
else:
    print("one.py is being imported into another module")
two.py
1
2
3
4
5
6
7
8
9
import one

print("top-level in two.py")
one.func()

if __name__ == "__main__":
    print("two.py is being run directly")
else:
    print("two.py is being imported into another module")

Running python one.py returns: top-level in one.py one.py is being run directly

While running python two.py returns: top-level in one.py one.py is being imported into another module top-level in two.py func() in one.py two.py is being run directly

Artificial Intelligence Nano Degree

| Comments

I started the Udacity Artificial Intelligence Nano Degree last month.

Project 1, Sudoku Solver

The Sudoku Solver was a fun project. I enjoy learning by writing code to make tests pass, like Ruby Katas.

Project 2, Isolation Game

Creating an algo to play an Isolation Game is much more complicated than the first project and was very frustrating to build. Unlike the first project, there is barely a test framework to follow. You push your methods to a remote Udacity server which gives feedback about test failures. This development cycle is difficult to debug because you can neither use a debugger nor get print output. After the minimax and alphabeta functions pass the remote tests, things get interesting. You are asked to write custom heuristics in order to decide the best move for your player. I wish Udacity gave more hints as to game heuristics.

Writing a synopsis for a research paper is also required for the project. I chose to write about AlphaGo. Due to my lack of knowledge reading math proofs, it took several attempts to understand the paper.

Project 3, Implement a Planning Search

Using planning problems—states, actions, and goals for planning algorithms. I enjoyed this unit of the course and it was interesting to study the development of Graph Plan algorithms since the 1970’s.

Project 4, Hidden Markov Models

The objective of this unit is to create an algo that can correctly guess sign language words. Hidden Markov models are interesting and remind me of deep learning algos. They both seem to solve a similar problem set.

Overall Thoughts

I am not sure this course is worth the $800 price. The class was interesting, but very time consuming. The benefits to taking the Udacity course over self learning on the internet is the Slack channel, forum and project feedback. I used the forum when I was stuck on a problem. I did not take advantage of the Slack channel. The project feedback was helpful. I will not be taking the next unit in the Artifical Intelligence Nano Degree. Instead, I am progressing through the Fast AI classes. I plan on writing a post after I finish the course.

My Photo Storage Solution

| Comments

I have been searching for a photo storage solution. My solution only involves two devices, a raspberry pi and an external hard drive. This post will explain how to set up your pi in order to automatically transfer RAW and JPG photos from an SD card to the hard drive connected to the pi. The scripts I used are available here.

I like this solution because the entire process is automatic. You are notified when the photo transfer has finished with an email. Also, for redundancy, I created a cron job that syncs all of the photos on my hard drive to Amazon photos. I use this service to sync the photos in the cloud.

Steps

Mount the external hard drive to a folder on your pi.
  • sudo fdisk -l take note of the Device, something like /dev/sdb1
  • I like to mount the drive under the home dir, sudo mount /dev/sdb1 ~/external_hd/
  • Run lsblk -f to know here files are mounted
Create a udev rule
  • udev rules live here: /etc/udev/rules.d
  • cd /etc/udev/rules.d
  • touch 50-sdcard.rules
  • chmod +x 50-sdcard.rules
  • sudo fdisk -l again to find the sd card device name, in this instance ‘sda1’
  • echo 'KERNEL=="sda1", ACTION=="add", RUN+="/home/pi/sdcard/sdcard_added.sh"' < 50-sdcard.rules
  • tail -f /var/log/syslog is extremely useful for debugging udev rules
Create an fstab privilege
  • vi /etc/fstab
  • Find device uuid with blkid
  • Add UUID=<your-uuid> /home/pi/SD_CARD auto rw,users,noauto 0 0
  • Uncomment out ‘user_allow_other’ in /etc/fuse.conf
rsync when an sd card is added
  • touch /home/pi/sdcard/sdcard_added.sh
  • chmod +x /home/pi/sdcard/sdcard_added.sh
  • create the script
1
2
3
4
5
#!/bin/bash

echo "`date +%Y-%m-%d:%H:%M:%S` SD card inserted" >> /home/pi/sdcard/output.log
mount /dev/sda1 /home/pi/SD_CARD
rsync -av /home/pi/SD_CARD/DCIM/100MSDCF/ /home/pi/external_hd/raw_photos
Sync photos to cloud provider
  • Set up cron job
  • sudo crontab -e
  • 00 4 * * * /home/pi/sd_card/rclone_cron.sh, this runs the cron job at 4 am, daily
  • create the bast script
1
2
#!/bin/sh
/usr/sbin/rclone sync /home/pi/external_hd/raw_photos/ amazon:raw_photos --config /home/pi/.rclone.conf -v
Send Email Notification
  • I used this post to easily add email notifications when the rsync process completes
Tips Since Creating My Photo Process
  • I could not get the above code to work with Debian due to permission issues
  • It is better to identify from UUID instead of KERNEL for setting up udev rules, run ls -l /dev/disk/by-uuid to discover device UUID’s
RClone
  • Amazon has blocked all thrid party applications from pushing to Amazon Drive. For more information see this
  • I am going to implement Duplicati as a solution to sync my photos to Amazon Drive
  • Another potential solution is to continute to use rclone, but host the files on Google Drive

RSpec and Elastic Search

| Comments

I had a difficult time setting up ElasticSearch on both RSpec (CircleCI) and Heroku. The ElasticSearch test cluster was not working on the CircleCI Docker image. Fortunately, one can configure the Circle environment to start with an ElasticSearch process. So, instead of using the test cluster, both my local testing environment and Circle environment use a real ElasticSearch process.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#
# ElasticSearch
#

config.before :all, elasticsearch: true do
  port = ENV['CIRCLE_CI_ES_URL'].present? ? 9200 : 9250
  Elasticsearch::Model.client = Elasticsearch::Client.new(port: port)
end

config.before :each, elasticsearch: true do
  Campaign.__elasticsearch__.create_index!(force: true)
end

config.after :each, elasticsearch: true do
  Campaign.__elasticsearch__.delete_index!
end

When a test needs to use ElasticSearch:

1
2
3
4
5
before { Campaign.__elasticsearch__.refresh_index! }

describe '#search', elasticsearch: true do
  expect(search.results).to be_present
end

I ran into issues using ElasticSearch on Heroku when creating an index. Heroku review apps are configurable by defining an app.json. In the json file, Heroku can spin up an ElasticSearch process.

1
2
3
4
5
6
7
8
9
10
11
12
13
"scripts": {
  "postdeploy": "bundle exec rake db:schema:load db:seed"
},
"formation": {
  "web": {
    "size": "free",
    "quantity": 1
  },
  "elasticsearch": {
    "size": "free",
    "quantity": 1
  }
}

The process is running before any Ruby code is executed. The next step is to create the index using postdeploy. Before creating ActiveRecord objects in the seed.rb file, create an index with Model.index(force: true).

Vim, Thoughts After Two Weeks

| Comments

Purposefully, I began using Vim during a non-strenuous work week which helped manage frustration. I have been using solely Vim as a text editor for two weeks and have experienced improved efficienies over my previous text editor, Sublime.

Improvments

The ability to change between files quickly using the Ctrl-P plugin is the biggest quality of life increase. Commands such as / and ? make finding word matches in a file easy. Using RSpec with Vim is also great; I can change my code and re-run a single test without switching screens! There are many small wins with using Vim and they add up.

Pain Points

Adding new files does not work if the directory does not exist. The mkdir plugin solves this issue. Furthermore, commenting out code is a pain without vim-commentary. Checking my vimrc file into version controller has been helpful in debgging where breaking changes originate.

More Resources

Vim Awesome

Level Up With Vim

| Comments

How much time do you spend in your text editor? We spend most of our time reading through text and jumping between files. My goal is to find a tool in order to help me do this more efficiently. I have found that learning Vim is not overwhelming or difficult if you break the units of work down into manageable pieces. The ultimate goal is to use Vim fulltime. Below are my curated steps:

Intro Video
Vimtutor
  • Comprehensive introduction to Vim
  • Type vimtutor in a Terminal session
  • Complete a few sections per day
Screencasts
Advanced
Helpful Links

Filtering Ack Results

| Comments

While setting up my VIM environment, I read a blogpost which states ack can ignore directories. For cleaner ack output you can setup the file below:

.ackrc
1
2
3
4
--ignore-dir=log
--ignore-dir=public/assets
--ignore-dir=vendor/assets
--ignore-dir=tmp/cache

Rails `before_action` Method

| Comments

I did not know you could give a before_action a block argument. This is a benefit because you do not have to create a ruby method and pass the method into the before_action.

This got me thinking, what does before_action look like in Rails? The following is the source code from Rails 4.2.7, from AbstractController::Callbacks::ClassMethods.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# :method: before_action
#
# :call-seq: before_action(names, block)
#
# Append a callback before actions. See _insert_callbacks for parameter details.

# Take callback names and an optional callback proc, normalize them,
# then call the block with each callback. This allows us to abstract
# the normalization across several methods that use it.
#
# ==== Parameters
# * <tt>callbacks</tt> - An array of callbacks, with an optional
#   options hash as the last parameter.
# * <tt>block</tt>    - A proc that should be added to the callbacks.
#
# ==== Block Parameters
# * <tt>name</tt>     - The callback to be added
# * <tt>options</tt>  - A hash of options to be used when adding the callback

def _insert_callbacks(callbacks, block = nil)
  options = callbacks.extract_options!
  _normalize_callback_options(options)
  callbacks.push(block) if block
  callbacks.each do |callback|
    yield callback, options
  end
end

[:before, :after, :around].each do |callback|
  define_method "#{callback}_action" do |*names, &blk|
    _insert_callbacks(names, blk) do |name, options|
      set_callback(:process_action, callback, name, options)
    end
  end
end

And I can do cool things like:

1
2
3
before_action -> {
  @foo = Model.find params[:bar]
}

I would like to point out the set_callback method.

1
2
3
4
before_action :authenticate

# can also be called like this
set_callback :process_action, :before, :authenticate

Extend and Include in Ruby

| Comments

I have been trying to clean up some old code with Ruby modules. This post is to help me remember the differences between include and extend in Ruby.

1
2
3
class Foo
  extend ActionView::Helpers::NumberHelper
end
1
2
3
module Foo
  extend ActionView::Helpers::NumberHelper
end
1
2
Foo.number_to_currency 2
 => "$2.00"
1
2
3
class Foo
  include ActionView::Helpers::NumberHelper
end
1
2
3
foo = Foo.new
foo.number_to_currency 2
 => "$2.00"