How to create a Ruby Gem: Publishing

This is the third and final part of the “How to create a Ruby Gem series” where we are going to finalize our plugin and publish it to RubyGems.

The following topics will be covered:

• Reading and processing a ZIP archive using the rubyzip module.
• Testing ZIP processing.
• Uploading/downloading files via Lokalise API.
• Using VCR to record HTTP interactions and replay them on subsequent test runs.
• Publishing your gem to the RubyGems platform.

 

Below are the previous parts of the series:

• First part
• Second part

Importer class

Import method

First of all, let’s finalize our import! method in the lib/lokalise_rails/task_definition/importer.rb file. This method should download translation files from Lokalise and store them in our Rails application.

def import!
  unless proceed_when_safe_mode?
    $stdout.print 'Task cancelled!'
    return false
  end

  open_and_process_zip download_files['bundle_url']

  $stdout.print 'Task complete!'
  true
end

This method is going to check whether or not the locales directory is empty. If it is not empty, we’ll ask the user whether s/he would like to proceed using the proceed_when_safe_mode? method. This check won’t be performed if the “safe mode” option is disabled.

Next, we’ll open the URL pointing to the archive containing all the translation files. After extracting the files and placing them inside the locales folder, we’ll output a “Task complete!” message and return true as a result.

Let’s also add the proceed_when_safe_mode? method:

def proceed_when_safe_mode?
  return true unless LokaliseRails.import_safe_mode && !Dir.empty?(LokaliseRails.locales_path.to_s)

  $stdout.puts "The target directory #{LokaliseRails.locales_path} is not empty!"
  $stdout.print 'Enter Y to continue: '
  answer = $stdin.gets
  answer.to_s.strip == 'Y'
end

This is going to check whether safe mode is enabled and whether the locales directory is empty. If the answer to both is yes, then it will ask the user to confirm the import operation. Great!

Opening and reading a ZIP file

Now let’s see how to open a ZIP file containing our translations. Create a new method:

def open_and_process_zip(path)
  Zip::File.open_buffer(open_file_or_remote(path)) do |zip|
    fetch_zip_entries(zip) { |entry| process!(entry) }
  end
end

This method accepts a path to the archive (in theory, this path may point to a local or a remote file) and then utilizes the rubyzip module to open it. Then each entry is processed one by one.

Create yet another method to read either a local or remote resource:

def open_file_or_remote(path)
  parsed_path = URI.parse(path)
  if parsed_path&.scheme&.include?('http')
    parsed_path.open
  else
    File.open path
  end
end

While you could proceed without creating this method and utilize Kernel open instead, this is not safe and not recommended for production use.

The next step is the fetch_zip_entries method:

def fetch_zip_entries(zip)
  return unless block_given?

  zip.each do |entry|
    next unless proper_ext? entry.name

    yield entry
  end
end

Here we are fetching files in the archive one by one and checking their extensions. If the extension is valid, then we process the file; otherwise, we just skip it.

We will need the proper_ext? method when coding another class as well, so let’s define it for the Base class within the lib/lokalise_rails/task_definition/base.rb file:

require 'pathname'

module LokaliseRails
  module TaskDefinition
    class Base
      class << self
        private

        def proper_ext?(raw_path)
          path = raw_path.is_a?(Pathname) ? raw_path : Pathname.new(raw_path)
          LokaliseRails.file_ext_regexp.match? path.extname
        end
      end
    end
  end
end

This method converts the path to a Pathname and makes sure its extension matches the regular expression stored in the file_ext_regexp option.

Processing ZIP file entries

Finally, let’s create the process! method inside the importer.rb file:

def process!(zip_entry)
  data = YAML.safe_load zip_entry.get_input_stream.read # <====== 1
  subdir, filename = subdir_and_filename_for zip_entry.name # <====== 2
  full_path = "#{LokaliseRails.locales_path}/#{subdir}"
  FileUtils.mkdir_p full_path # <====== 3

  File.open(File.join(full_path, filename), 'w+:UTF-8') do |f| # <====== 4
    f.write data.to_yaml
  end
rescue StandardError => e
  $stdout.puts "Error when trying to process #{zip_entry&.name}: #{e.inspect}" # <====== 5
end

This method does the following:

1. Reads the file contents fetched from the archive.
2. Gets the filename and its directory name from within the archive. For example, the en.yml file may be stored in the en folder. This structure has to be preserved inside the Rails app as well, therefore we are fetching both the filename and directory.
3. Creates full path to the translation file, including the locales folder, file directory, and its name. Then creates the corresponding path.
4. Opens the newly created file and pastes translation data into it.
5. If something goes wrong, outputs an error message.

Add yet another method to fetch the file and directory name. Place it in the base.rb file:

def subdir_and_filename_for(entry)
  Pathname.new(entry).split
end

This method will return a plain array with two items: directory and filename.

Downloading translations

So, we are done with processing the ZIP archive. The last step is to actually download the requested translations from Lokalise. To achieve this, create a new download_files method inside importer.rb:

def download_files
  opts = LokaliseRails.import_opts

  api_client.download_files LokaliseRails.project_id, opts
rescue StandardError => e
  $stdout.puts "There was an error when trying to download files: #{e.inspect}"
end

We are reading the import options, and then using the ruby-lokalise-api client to perform the actual download.

Add the api_client method to base.rb:

require 'pathname'
require 'ruby-lokalise-api'

module LokaliseRails
  module TaskDefinition
    class Base
      class << self
        def api_client
          @api_client ||= ::Lokalise.client LokaliseRails.api_token
        end

        # ...
      end
    end
  end
end

The last step is to load all the necessary modules in the importer.rb:

require 'zip'
require 'yaml'
require 'open-uri'
require 'fileutils'

At this point, the import feature is done and dusted!

Writing tests for the import task

To make sure everything is working well, let’s write a test for the import feature within the spec/lib/tasks/import_task_spec.rb file (remove the old test we added in the previous part):

RSpec.describe LokaliseRails do
  let(:loc_path) { described_class.locales_path }
  let(:remote_trans) { 'https://github.com/bodrovis/lokalise_rails/blob/master/spec/dummy/public/trans.zip?raw=true' }

  it 'import rake task downloads ZIP archive properly' do
    allow(LokaliseRails::TaskDefinition::Importer).to receive(
      :download_files
    ).and_return(
      {
        'project_id' => '123.abc',
        'bundle_url' => remote_trans
      }
    )

    expect(import_executor).to output(/complete!/).to_stdout

    expect(LokaliseRails::TaskDefinition::Importer).to have_received(:download_files)
    expect(count_translations).to eq(4)
    expect_file_exist loc_path, 'en/nested/main_en.yml'
    expect_file_exist loc_path, 'en/nested/deep/secondary_en.yml'
    expect_file_exist loc_path, 'ru/main_ru.yml'
  end

  # ...
end

In this test we are trying to download a sample archive and make sure four translation files are being created. However, we also need to add before and after hooks:

before do
  mkdir_locales
  rm_translation_files
end

after :all do
  rm_translation_files
end

Prior to running any tests, we have to make sure the locales directory is actually created and that it is empty. Then, after all the tests are executed, we will perform a cleanup by removing all translation files.

Define some new helper methods in the spec/support/file_manager.rb:

def expect_file_exist(path, file)
  file_path = File.join path, file
  expect(File.file?(file_path)).to be true
end

def locales_dir
 Dir["#{LokaliseRails.locales_path}/**/*"]
end

def mkdir_locales
  FileUtils.mkdir_p(LokaliseRails.locales_path) unless File.directory?(LokaliseRails.locales_path)
end

def rm_translation_files
  FileUtils.rm_rf locales_dir
end

def count_translations
  locales_dir.count { |file| File.file?(file) }
end

Now you can run rspec . and make sure your tests are running properly!

Testing a third-party API with VCR

The final thing I would like to do is to test that the archive is being downloaded properly from Lokalise. We perform the download operation via the API, and theoretically the same operation can be executed within our test. However, I would rather not send real API requests each time the tests are executed. Instead, it would be nice to record the API interaction once, store the result within a given file, and then “replay” this interaction on subsequent test runs. To achieve this, we are going to use a solution called VCR.

We have already added it to the gemspec, but it requires some additional configuration. Therefore, create a new spec/support/vcr.rb file:

require 'vcr'

VCR.configure do |c|
  c.ignore_hosts 'codeclimate.com'
  c.hook_into :faraday
  c.cassette_library_dir = File.join(File.dirname(__FILE__), '..', 'fixtures', 'vcr_cassettes')
  c.filter_sensitive_data('<LOKALISE_TOKEN>') { ENV.fetch('LOKALISE_API_TOKEN') }
  c.configure_rspec_metadata!
end

VCR stores all HTTP interactions in special YAML files which are called cassettes (if you are 25 or older, you probably remember VCR cassettes with films like “Terminator” or “Conan the Barbarian”). However, certain data like API tokens should not be stored in the cassette, therefore we will use filter_sensitive_data to exclude it.

Now create a new spec/lib/lokalise_rails/task_definitions/importer_spec.rb file:

describe LokaliseRails::TaskDefinition::Importer do
  it 'returns a proper download URL' do
    project_id = 'PROJECT_ID'
    allow(LokaliseRails).to receive(:project_id).and_return(project_id)

    response = VCR.use_cassette('download_files') do
      described_class.download_files
    end

    expect(LokaliseRails).to have_received(:project_id)
    expect(response['project_id']).to eq(project_id)
    expect(response['bundle_url']).to include('s3-eu-west-1.amazonaws.com')
  end
end

Make sure to replace PROJECT_ID with a real Lokalise project ID. In this test we are using a cassette called download_files. If said cassette does not yet exist, it will be created for you by sending a real API request. On subsequent runs, however, the recorded interaction will be utilized.

Exporter class

Export method

Now let’s take care of the export! method in the lib/lokalise_rails/task_definition/exporter.rb file. It should upload all translation files matching the given criteria to Lokalise:

require 'base64' # <===== 1

module LokaliseRails
  module TaskDefinition
    class Exporter < Base
      class << self
        def export!
          queued_processes = []
          each_file do |full_path, relative_path| # <===== 2
            queued_processes << api_client.upload_file( # <===== 3
              LokaliseRails.project_id, opts(full_path, relative_path)
            )
          rescue StandardError => e
            $stdout.puts "Error while trying to upload #{full_path}: #{e.inspect}"
          end

          $stdout.print 'Task complete!'

          queued_processes # <===== 4
        end
      end
    end
  end
end

1. Import the base64 module which we are going to use to properly encode the translation file contents.
2. Take each file from within the locales directory.
3. Upload a translation file to Lokalise via the API. The upload process will take place in the background, so the API will respond with queued process data.
4. Return an array with all the queued processes.

Processing translation files

Add a new each_file method to the exporter.rb:

def each_file
  return unless block_given?

  loc_path = LokaliseRails.locales_path
  Dir["#{loc_path}/**/*"].sort.each do |f|
    full_path = Pathname.new f

    next unless file_matches_criteria? full_path

    relative_path = full_path.relative_path_from Pathname.new(loc_path)

    yield full_path, relative_path
  end
end

This method iterates over the files in the locales directory and keeps only those that match the criteria (we’ll take care of the corresponding method in a moment). Then for each file, we get its path as it relates to the locales directory, for example: if the file is named ~/my_project/config/locales/en/nested/en.yml, the relative path will be en/nested/en.yml. We will send this relative path to Lokalise thus preserving the original file structure.

Here’s the method to check whether or not the file should be processed:

def file_matches_criteria?(full_path)
  full_path.file? && proper_ext?(full_path) &&
    !LokaliseRails.skip_file_export.call(full_path)
end

So, we have to make sure that:

1. The entry is actually a file, not a directory.
2. The entry has the proper extension (we already created the proper_ext? method earlier).
3. The file was not blacklisted using the skip_file_export option. This option accepts a lambda or a procedure which should return either true or false depending on the filename.

File upload options

To finalize the export feature, add the opts method which should return the below file upload options:

def opts(full_p, relative_p)
  content = File.read full_p # <===== 1

  lang_iso = YAML.safe_load(content)&.keys&.first # <===== 2

  initial_opts = {
    data: Base64.strict_encode64(content.strip), # <===== 3
    filename: relative_p, # <===== 4 
    lang_iso: lang_iso # <===== 5
  }

  initial_opts.merge LokaliseRails.export_opts # <===== 6
end

1. Read translation file contents.
2. Try to determine the language ISO code of the given translation file.
3. Encode translations using the base64 module.
4. Provide the relative path to the translation file as its name.
5. Set the language ISO code.
6. Add any additional export options the user has provided.

Testing the exporter

Next, let’s test our exporter functionality within spec/lib/lokalise_rails/task_definitions/exporter_spec.rb. First of all, add some let instructions and hooks:

describe LokaliseRails::TaskDefinition::Exporter do
  let(:filename) { 'en.yml' }
  let(:path) { "#{Rails.root}/config/locales/nested/#{filename}" }
  let(:relative_name) { "nested/#{filename}" }

  before :all do
    add_translation_files!
  end

  after :all do
    rm_translation_files
  end
end

Create two new helper methods within spec/support/file_manager.rb:

def add_translation_files!
  FileUtils.mkdir_p "#{Rails.root}/config/locales/nested"
  File.open("#{Rails.root}/config/locales/nested/en.yml", 'w+:UTF-8') do |f|
    f.write en_data
  end
end

private

def en_data
  <<~DATA
    en:
      my_key: "My value"
      nested:
        key: "Value 2"
  DATA
end

Return to the spec file and add the following test:

it 'sends a proper API request' do
  allow(LokaliseRails).to receive(:project_id).and_return('PROJECT_ID')

  process = VCR.use_cassette('upload_files') do
    described_class.export!
  end.first

  expect(process.project_id).to eq(LokaliseRails.project_id)
  expect(process.status).to eq('queued')
end

Don’t forget to replace PROJECT_ID with a real Lokalise project ID. Now you can run rspec . and observe the results!

Publishing to RubyGems

So, my congratulations to you: our Ruby Gem is now ready to be deployed to RubyGems! Before doing this, you might want to add some more tests to increase test coverage; all specs for the lokalise_rails gem can be found at GitHub. Also, I would recommend pushing all the changes to your GitHub repo and making sure that the TravisCI tests are all green as well. Finally, run Rubocop and fix the issues found.

Then, navigate to rubygems.org and sign up (it’s free).  Next, inside your command line interface, make sure you have the up-to-date RubyGems software and Bundler:

gem update --system
gem install bundler

Build your gem using gemspec:

gem build lokalise_rails.gemspec

The above command is going to create a new file with a *.gem extension named after your gem and its version. The final step is to actually publish your gem:

gem push lokalise_rails.VERSION.gem

Make sure to publish the *.gem file, not the gemspec! You will be asked to log in via RubyGems, and then after a few seconds your new gem will be available!

Test the installation by running:

cd ..
gem install lokalise_rails

Also you can view your gem’s information by visiting rubygems.org/gems/lokalise_rails.

Great job!

Conclusion

We have reached the end of both this article and the whole series. We have discussed how to create a Ruby Gem from scratch, add all the necessary configurations, how to introduce a testing suite, and lastly how to publish it to RubyGems. As a final note: don’t be shy about creating a new gem, even if it seems too simple. The open source world is all about contributions from different developers from around the globe, and even if your plugin can help ten people, that’s still a great achievement!

So, that’s all for today, folks. Thank you for staying with me to the end and see you really soon!

Return to the second part