Browse Source

Revamping biogems

remotes/origin/dependabot/bundler/haml-5.1.2
Pjotr Prins 5 years ago
parent
commit
9e7374e199
9 changed files with 93 additions and 88 deletions
  1. +1
    -0
      .gitignore
  2. +0
    -1
      Gemfile
  3. +39
    -21
      INSTALL.md
  4. +10
    -27
      README.md
  5. +1
    -1
      Rakefile
  6. +7
    -9
      bin/fetch-geminfo.rb
  7. +27
    -0
      bin/fetch-gemlist.rb
  8. +0
    -21
      bin/fetch-src.rb
  9. +8
    -8
      features/support/env.rb

+ 1
- 0
.gitignore View File

@@ -8,3 +8,4 @@ build/
data/
var/
website/
.sass-cache/

+ 0
- 1
Gemfile View File

@@ -15,7 +15,6 @@ group :development do
gem "rake"
gem "yard", "~> 0.7"
gem "rspec", "~> 2.5"
gem "bundler", "~> 1.3"
gem "nokogiri", "~> 1.5"
gem "cucumber", "~> 1.2"
gem "thread", "~> 0.1.3"


+ 39
- 21
INSTALL.md View File

@@ -1,42 +1,60 @@
# INSTALL

Biogem.info is a middleman generated website.
It comes with a number of scripts and rake tasks which generate
the information displayed on the site in YAML format (usually to STDOUT).
Biogem.info uses haml + sass to generate a website.

## Installation

You will need libxml support, e.g.
The site generation consists of a number of separate steps
which can easily be tested independently.

apt-get install libxslt-dev libxml2-dev
## Installation

Install Ruby with openssl support
To avoid rvm+bundler hell, the preferred route is to use GNU Guix and
set the environment with my
[ruby-guix-env](https://github.com/pjotrp/guix-notes/blob/master/scripts/ruby-guix-env)
script. E.g.

rvm pkg install openssl
rvm pkg install iconv
rvm remove 1.9.2
rvm install 1.9.2 -C --with-openssl-dir=$HOME/.rvm/usr,--with-iconv-dir=$HOME/.rvm/usr
guix install ruby rake
. ruby-guix-env
gem install haml sass

Checkout the source code and run bundler. See also the create_data.sh.
To run the cucumber tests

bundle update
cucumber features/

To run the cucumber tests
A 'quick' test run

bundle exec cucumber features/
rake -- --test

To run a 'quick' test run
## Generating the website

bundle exec rake -- --test
### Fetch gems from rubygems.org

which will generate the site with just a few biogems. Next run
The first step is to fetch relevant gems from http://rubygems.org/. This
is done with

bundle exec middleman server
./bin/fetch-gemlist.rb

## Trouble shooting
## Troubleshooting

If you get an error

./bin/fetch-geminfo.rb:163:in `block in <main>': undefined method `authors'

run the script with 'bundle exec' prepended.

### GitHub API access limits

Without using authentication, the GitHub API allows only 60 requests
per hour from a single IP address. But during the data collection
phase of generating the biogems.info website, the script currently
needs to make around 200 requests to this API to fetch the number of
issues and stargazers for each gem.

To get around this, go to the applications tab on your GitHub
settings page and generate a new "Personal API access token". Then
copy that token (but not into the repository!), and before running the ./create_data.sh script,
set the GITHUB_API_TOKEN environment variable like this:

export GITHUB_API_TOKEN="copy-here-the-token-string-from-github"

That lets the script make 5000 requests per hour, which should be
more then enough.

+ 10
- 27
README.md View File

@@ -1,23 +1,23 @@
= Biogems.info
# Biogems.info

http://biogems.info/ tracks interesting Ruby gems for bioinformatics.

The following information describes the steps to getting your biogem
registered on biogems.info. If you are looking to generate the
biogems.info website instead: read INSTALL.md!

http://biogems.info tracks interesting Ruby gems for bioinformatics.

To get your gem listed, simply create a gem named 'bio-mygem', i.e.
start the name with bio dash, and push it onto rubygems.org. You can
start the *name* with bio dash, and push it onto rubygems.org. You can
use the biogem tool to create the plumbing, if you like. If you wish
to name your gem differently, or host the gem elsewhere, it can still
be listed. Either add a project description to /etc/biogems/name.yaml,
or add an issue to the github tracker at
https://github.com/pjotrp/biogems.info/issues

= Gem settings
# Displayed gem information

The settings displayed, on biogems.info, are the ones you specify
for a gem. In particular
in your gem. In particular

gem.name = "bio-gem"
gem.homepage = "http://github.com/helios/bioruby-gem"
@@ -28,7 +28,7 @@ for a gem. In particular
gem.email = "ilpuccio.febo@gmail.com"
gem.authors = ["Raoul J.P. Bonnal"]

This is the information pushed to http://rubygems.org when releasing a
This is the information pushed to http://rubygems.org/ when releasing a
gem. biogems.info harvests this information, with the download statistics.

= Additional settings
@@ -50,28 +50,11 @@ It is also possible the overriding gemname.yaml is in your github
repository(!) Just tell us where it resides through
https://github.com/pjotrp/biogems.info/issues

= GitHub API access limits

Without using authentication, the GitHub API allows only 60 requests
per hour from a single IP address. But during the data collection
phase of generating the biogems.info website, the script currently
needs to make around 200 requests to this API to fetch the number of
issues and stargazers for each gem.

To get around this, go to the applications tab on your GitHub
settings page and generate a new "Personal API access token". Then
copy that token (but not into the repository!), and before running the ./create_data.sh script,
set the GITHUB_API_TOKEN environment variable like this:

export GITHUB_API_TOKEN="copy-here-the-token-string-from-github"

That lets the script make 5000 requests per hour, which should be
more then enough.

= Website source
# Website source

This repository on github contains the source code for the
http://biogems.info website.
http://biogems.info/ website.

Biogems.info is an initiative by the BioRuby developers
Copyright (C) 2011,2012,2013,2014 Pjotr Prins <pjotr.prins@thebird.nl>
Copyright (C) 2011-2015 Pjotr Prins <pjotr.prins@thebird.nl>

+ 1
- 1
Rakefile View File

@@ -45,7 +45,7 @@ task :biogems do |t|
end

task :default => [ :biogems ] do
`bundle exec middleman build`
`middleman build`
end



+ 7
- 9
bin/fetch-geminfo.rb View File

@@ -17,8 +17,11 @@ include BioGemInfo::GitHub

IS_NEW_IN_DAYS = 7*6 # 6 weeks

$stderr.print "GITHUB TOKEN = ",ENV['GITHUB_API_TOKEN'],"\n"

token = ENV['GITHUB_API_TOKEN']
if not token or token == ''
raise "Github token is missing"
end
$is_debug = ARGV.index('--debug')
is_testing = ARGV.index('--test')

@@ -27,8 +30,6 @@ print "# Using Ruby ",RUBY_VERSION,"\n"

projects = Hash.new

list = x.json

# Return the working URL, otherwise nil
def check_url url
if url =~ /^http:\/\/github/
@@ -113,13 +114,10 @@ list_in_random_order.each do | name |
$stderr.print name,"\n" if $is_debug
info = Hash.new
# Fetch the gem YAML definition of the project
$stderr.print "bundle exec gem specification -r #{name.strip}\n" if $is_debug
fetch = `bundle exec gem specification -r #{name.strip}`
$stderr.print "gem specification -r #{name.strip}\n" if $is_debug
fetch = `gem specification -r #{name.strip}`
if fetch != ''
spec = YAML::load(fetch)
# print fetch
# p spec
# When you get undefined method `authors' for #<Syck::Object:0x000000028cf3f0> (NoMethodError) make sure you try with 'bundle exec' instead!
info[:authors] = spec.authors
info[:summary] = spec.summary
info[:version] = spec.version.to_s


+ 27
- 0
bin/fetch-gemlist.rb View File

@@ -0,0 +1,27 @@
#!/usr/bin/env ruby
#
# This script fetches information from the gem
# tool

$: << "lib"

require 'json'
require 'yaml'
require 'net/http'
require 'uri'
require 'biogems'

$is_debug = ARGV.index('--debug')
is_testing = ARGV.index('--test')

$stderr.print "Get all rubygems starting with bio- (bio dash)\n"
if is_testing
list = ['bio','bio-blastxmlparser']
else
list = `gem search bio-`.split(/\n/)
list = list.map { |item| item.split(" ")[0] }
end



print list.to_json # output to STDOUT

+ 0
- 21
bin/fetch-src.rb View File

@@ -1,21 +0,0 @@
#! ruby

require 'yaml'
DIR='pkg'

fn = './var/biogems.yaml'

list = YAML::load(File.read(fn))

Dir.chdir(DIR) do
list.each do | name, info |
p name
git = info[:source_code_uri]
git = info[:homepage_uri] if !git
if not File.directory?(File.basename(git))
Kernel.system("git clone "+git)
else
print "Skipping ",git,"\n"
end
end
end

+ 8
- 8
features/support/env.rb View File

@@ -1,11 +1,11 @@
require 'bundler'
begin
Bundler.setup(:default, :development)
rescue Bundler::BundlerError => e
$stderr.puts e.message
$stderr.puts "Run `bundle install` to install missing gems"
exit e.status_code
end
# require 'bundler'
# begin
# Bundler.setup(:default, :development)
# rescue Bundler::BundlerError => e
# $stderr.puts e.message
# $stderr.puts "Run `bundle install` to install missing gems"
# exit e.status_code
# end

$LOAD_PATH.unshift(File.dirname(__FILE__) + '/../../lib')
require 'bioinfo'


Loading…
Cancel
Save