DevHeads.net

Postings by Gerald Bauer

Vienna.rb Talk Notes: Mining for Gold Using the World's #1 and Most Popular Data Format (w/ Ruby and CsvReader)

Hello,

the talk notes from yesterday's Vienna.rb meetup titled Mining for
Gold Using the World's #1 and Most Popular Data Format (w/ Ruby and
CsvReader) [1].

tabreader library / gem v1.0 - read in tabular datafiles in text in the tabular (TAB) format

Hello,

I've published the tabreader library / gem version 1.0 [1]
that lets you read in tabular datafiles in text in the tabular (TAB) format.

Yes, it's as simple as:

line = "1\t2\t3"
values = line.split( "\t" )
pp values
# => ["1","2","3"]

or the "magic" packaged up in `TabReader`:

line = "1\t2\t3"
values = Tab.parse_line( line ) ## or TAB.parse_line or
TabReader.parse_line
pp values
# => ["1","2","3"]

or use the convenience helpers:

txt <<=TXT
1\t2\t3
4\t5\t6
TXT

records = Tab.parse( txt ) ## or TAB.parse or TabReader.pars

csvjson library / gem v1.0 - read tabular data in the CSV <3 JSON format, that is, comma-separated values CSV (line-by-line) records with JSON encoding rules

Hello,

I've published a new library / gem, that is, csvjon that lets you
read tabular data in the (new) CSV <3 JSON format, that is,
comma-separated values CSV (line-by-line) records with javascript
object notation (JSON) encoding rules.

What's CSV <3 JSON?

CSV <3 JSON is a Comma-Separated Values (CSV)
variant / format / dialect
where the line-by-line records follow the
JavaScript Object Notation (JSON) encoding rules.
It's a modern (simple) tabular data format that
includes arrays, numbers, booleans, nulls, nested structures, comments and more.
Example:

# "Vanilla" CSV <3 JSON

Why the standard CSV library author deserves our hugs and thank yous

Hello,

I’ve written a new episode in the why the csv standard library is
broken, broken, broken (and how to fix it).

It’s a new introduction to the series titled “I apologize - Sorry,
Sorry, Sorry.” [1] to clarify that I am in no way blaming the author
(really I’m blaming YOU :-), that is, all the free-loaders waiting for
a miracle.

The author or the authors of the standard CSV library deserve our
hugs and thank yous for the great work and many hours (for sure many
unpaid and volunteered) put into the CSV library. We are all standing
on the shoulders of giants. Thank you.

Why the CSV standard library is broken (and how to fix it), Part IV or Numerics a.k.a. Auto-Magic Type Inference for Strings and Numbers?

Hello,

I've written a new (and fourth) episode on why the CSV standard library is
broken, broken, broken (and how to fix it).

Let's have a look at numerics a.k.a. auto-magic type inference for
strings and numbers [1].

Here's the challenge for the standard csv library.
Let's read data.csv:

1,2,3
"4","5","6"

Using these popular two rules (bonus for NaNs - not a number).

Rule 1: Use "un-quoted" values for float numbers e.g. 1,2,3 or 1.0,
2.0, 3.0 etc.

Rule 2: Use quoted values for "non-numeric" strings e.g.

csvreader v1.0 - read tabular comma-separated values (csv) the right way (incl. hash readers, converters, enumerables, dialects, and more)

Hello,

I've uploaded [version 1.0 of the new comma-separated values
csvreader library / gem [1] that lets you read tabular data in the
comma-separated values (csv) format the right way :-), that is, the
basic methods such as `Csv.read `or `CsvHash.read` use best practices
out-of-the-box with zero-configuration.

Under the hood the new library includes purpose-built "backend"
parsers (e.g.

Glim - A new faster jekyll website compiler clone / alternative (in ~3000 lines of ruby)

Hello,

Jekyll not fast enough? Why not build a compatible faster clone /
alternative from scratch / zero (in ruby)? [1] Sounds crazy?

Allan Odgaard (of TextMate fame) has just done it and with about 3
000 lines of ruby code offers even more features than jekyll itself
:-) e.g. built-in tags and categories for collections, lazy evaluation
and parallized builds, better defaults, and much more. Find out more
at the Glim website compiler source project repo [2]

Happy publishing w/ ruby. Cheers. Prost.

PS: By the way - did you know?

Why the CSV standard library is broken (and how to fix it), Part III or Returning a CSV Record as an Array? Hash? Struct? Row?

Hello,

I've written a new (and third) episode [1] in the series
why the CSV standard library is broken, broken, broken (and how to fix it).

Let's have a look at how to handle csv records. How about returning
an array? hash? struct? row? etc.

Or how about adding new "low-level" parsers for supporting more
dialects and formats or for faster code with c-extensions?

Questions and comments welcome. Cheers.

Ruby Conferences & Camps in 2018 Calendar Update @ Planet Ruby - What's Upcoming?

Hello,

I've updated the ruby conferences & camps in 2018 calendar [1].
What's upcoming?

October

- Sat 6 (1d) Ruby Russia @ Moscow, Russia
- Thu+Fri 25+26 (2d) RubyConf Malaysia @ Kuala Lumpur, Malaysia

November

- Fri 9 (1d) Keep Ruby Weird @ Austin, Texas, United States
- Tue-Thu 13-15 (3d) RubyConf @ Los Angeles, California, United States

Any others? Please tell. Cheers.

Why the CSV standard library is broken, broken, broken (and how to fix it), Part II or The Wonders of CSV Formats / Dialects

Hello,

I've written a follow-up [1] on why the CSV standard library is
broken, broken, broken. Let's have a look at the wonders of CSV
formats / dialects
such as MYSQL CSV export or PostgreSQL CSV or TEXT export.

In a nutshell - the standard CSV parser is a (simplistic) kludge
using string.split() and, thus, needs to get replaced with a "real" parser
that can handle more "edge" cases / formats.

Questions and comments welcome.

Also if want to see other CSV formats pre-configured and supported
"out-of-the-box" in the new csv reader, let us know.

Cheers.

kramup / kramdown-service v1.0.0 - Live Markdown Editor and HTTP JSON API w/ Sinatra v2

Hello,

I've updated the kramdown-service / kramup library and tool [1]
that lets you start your own local live markdown editor browser app and
json http api service w/ sinatra v2. Example:

Step 0 - Install the gem e.g.

$ gem install kramdown-service

Step 1 - Start the server / service e.g.

$ kramup

Step 2 - Open up the editor page in your browser e.g. use
`http://localhost:4567`.

That's it.

Enjoy. Cheers. Prost.

PS: Did you know?

EuRuKo (European Ruby Konference) 2018 @ Vienna (Austria) - 24+25th/Aug - Live Stream (Incl. All Talks)

Hello,

FYI: the EuRuKo 2018 just kicked off this morning in Vienna,
Austria. Big thanks to Ramón Huidobro (@SenorHuidobro), Super Princess
Pilar (@Tamacodechi) and the org team for making in happen.

Follow along with the EuRuKo 2018 live stream on YouTube for all
talks the next two days! [1].

Cheers. Prost.

[1] <a href="https://www.youtube.com/watch?v=YhPP_Knq0qM" title="https://www.youtube.com/watch?v=YhPP_Knq0qM">https://www.youtube.com/watch?v=YhPP_Knq0qM</a>

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

Why the CSV standard library is broken, broken, broken (and how to fix it)

Hello,

I’ve written up an article titled “Why the CSV standard library is
broken, broken, broken (and how to fix it)” [1].

Why not let’s make ruby the best it can be for data wrangling -
starting with the humble comma-separated values (CSV) format - the
world’s most popular data interchange format in text :-).

Try it yourself.

csvreader - read tabular data in the comma-separated values (csv) format the right way

Hello,

I've started to put together a new library / gem, that is, csvreader [1] -
that lets you read tabular data in the comma-separated values (csv) format
the right way :-), that is, uses best practices such as

- stripping / trimming leading and trailing spaces,
- skipping comments and blank lines,
- "fixes" quote errors and more

all out-of-the-box with zero-configuration.
And, thus, fixes some major bugs in the (old) standard csv library
with a purpose-built parser (instead of a supposed "faster" split(",") kludge).

Happy data wrangling with ruby. Cheers.

tabreader - read in tabular datafiles in text in the tab format; yes, uses values = line.split("/t")

Hello,

I've put together a new library / gem, that is, tabreader
that lets you read in tabular datafiles in text in the tab format;
yes, uses

values = line.split( "/t" )

or use the packaged up

TabReader.parse_line( ... )

or any of the convenience helpers incl.

csvrecord - read in comma-separated values (csv) records with typed structs / schemas

Hello,

I've put together a new library / gem called csvrecord [1] that
let's you read in comma-separated values (csv) records
with typed structs / schemas.

csvutils - tools 'n' scripts for working with comma-separated values (csv) datafiles - the world's most popular tabular data interchange format in text

Hello,

I've put together a new library / gem called csvutils [1] that collects
scripts for wrangling with comma-separated values (csv) datafiles -
the world's most popular tabular data interchange format in text.

Try the help option `-h/--help` with the command line tools.

csvpack Update - Work with tabular data packages in ruby (download, import into sql, query w/ ActiveRecord and more)

Hello,

I've updated the csvpack (*) library / gem [1] that
let's you work with tabular data packages in ruby.

Download comma-separated values (CSV) datafiles in text - also known
as tabular datafiles and read into any sql database (e.g. SQLite,
PostgreSQL, ...), query w/ ActiveRecord and more. Example:

require 'csvpack'

CsvPack.import(
's-and-p-500-companies',
'gdb'
)

Cheers.

sportweb Gem Update - Instant sport.db Web Admin (Rails Engine) Command Line Tool / Server - Browse All World Cups etc.

Hello,

for the world cup I've updated the sportweb command line tool [1] that
lets you browse your local sport.db / worldcup.db with a built-in
instant web admin server (and rails engine).

Browse all world cups (from Uruguay 1930 to Russia 2018) or build
your own sport league / tournament.

Cheers. Prost. Enjoy the beautiful game with ruby (and rails).

[1] <a href="https://github.com/sportdb/sport.db.web" title="https://github.com/sportdb/sport.db.web">https://github.com/sportdb/sport.db.web</a>

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

footty - football.db command line tool / client for world cup (russia 2018)

Hello,

for the upcoming football world cup in russia 2018 - the world’s
biggest show (3+ billion fans) - I’ve put together a little command
line tool / client in ruby called footty that lets you query the
online football.db HTTP JSON API services for upcoming or past
matches.

Building world-cup.json - Open (free) public domain world cup in Russia 2018 football datasets in JSON with Ruby (and the sportdb machinery)

Hello,

using the sportdb machinery (gems, command line tools and build
scripts in ruby) I've put together a [new free public domain dataset
for the (football) world cup in Russia 2018 [1].

Programming Blockchains Step-by-Step from Scratch (Zero) in Ruby. Starting with Crypto Hashes... - New Free (Online) Book @ Yuki & Moto Press

Hello,

I've added a new free (online) book to the Yuki & Moto Press
Bookshelf [1]. Let's welcome:

Programming Blockchains Step-by-Step from Scratch (Zero) in Ruby.
Starting with Crypto Hashes...

Ruby 3x3 - Ruby 3 Will Be 3 Times Faster - What’s News? Updates on Method JIT, Guilds, ...

Hello,

I've updated the Ruby 3x3 articles (and slides) collection [1] @ Planet Ruby.

What's News? Thanks to the RubyElixirConf Tawain new slide decks include:

o The State of / All About Guilds - A New Concurrency Model for Ruby
3 by Koichi Sasada, Cookpad

o Method JIT (Just-In-Time) Compiler for MRI (Matz's Ruby
Interpreter) - Optimizations in Ruby 2.6 Preview 1, 2 by Takashi
Kokubun, Treasure Data.

Anything missing? Additions welcome. Cheers.

(Re)building the Billion Dollar Ethereum World Computer from Scratch (Zero) in Ruby

Hello,

a little weekend experiment - let's (re)build the billion dollar
ethereum world computer from scratch (zero) using a 25-year-old
contract language - ruby :-) - and ye good 40-year-old SQL databases.

See a first live converted gold mine / token contract in universum
e.g.

Ruby Conferences & Camps in May 2018 - What's Upcoming?

Hello,

at the calendar page @ Planet Ruby we collect all ruby conferences
and camps from around the world. What's upcoming in May 2018?

- Rubyhack - May/3+4 (2d) Thu+Fri @ Salt Lake City, Utah, United States
- Ruby Unconf Hamburg - May/5+6 (2d) Sat+Sun @ Hamburg, Germany
- Ruby Open Source Software (ROSS) Conf Amsterdam (FREE) - May/11+12
(2d) Fri+Sat @ Amsterdam, the Netherlands
- Balkan Ruby - May/25+26 (2d) Fri+Sat @ Sofia, Bulgaria
- RubyKaigi - May/31-Jun/2 (3d) Thu-Sat @ Sendai, Miyagi, Japan

See all conferences & camps in 2018 [1]. Anything missing?

Ruby Conferences & Camps in April 2018 - What's Upcoming? Isle of Ruby, RubyConf Belarus, RubyConf Taiwan, ...

Hello,

at the calendar page @ Planet Ruby we collect all ruby conferences
and camps from around the world. What's upcoming in April 2018?

- Isle of Ruby - Apr/13-15 (3d) Fri-Sun @ Exeter, England, United Kingdom
- RailsConf - Apr/17-19 (3d) Tue-Thu @ Pittsburgh, Pennsylvania, United States
- RubyConf Belarus - Apr/21 (1d) Sat @ Minsk, Belarus
- Ruby X Elixir Conf Taiwan - Apr/27+28 (2d) Fri+Sat @ Taipei, Taiwan

See all conferences & camps in 2018 [1]. Anything missing? Updates
welcome. Cheers.

Bitcat - Bit catalog kitty browser - browse your digital bit(s) collections unchained (offline) w/ SQLite - a sinatra web app in ruby

Hello,

I've put together - bitcat - a little sinatra web app
(e.g.bitcat/app.rb [1]) in ruby packaged up as a command line tool
that lets you browse your digital bits(s) eg. crypto kitties and
friends unchained (offline) using a single-file SQLite database eg.
kitties.db.

Note: Bitcat includes the gene / genome reader in base32/kai
notation from copycats [2] - another ruby library for (crypto) cats /
kitties - and more.

Happy collecting. Happy gene mixing. Happy hacking with ruby. The
future is meow.

What's Your Hash Rate? Find out your Mining Speed w/ compute_hash_with_proof_of_work

Hello,

I've added a couple of new chapters to the "Programming Blockchains
Step-by-Step from Scratch (Zero)" [1] and starting with (crypto)
hashes...

What's News? Mining, mining, mining! What's your hash rate? Let's
find out and use the "stand-alone" version of the by now "classic"
compute_hash_with_proof_of_work function:

def compute_hash_with_proof_of_work( data, difficulty='00' )
nonce = 0
loop do
hash = Digest::SHA256.hexdigest( "#{nonce}#{data}" )
if hash.start_with?( difficulty )
return [nonce,hash] ## bingo!

Programming Blockchains Step-by-Step guide (in ruby). Let's build blockchains from scratch (zero). Let's start with crypto hashes...

Hello,

I've started a step-by-step guide that shows you how ot build
blockchains in ruby from scratch.

Starting with (crypto) hashes...

require 'digest'
Digest::SHA256.hexdigest( 'Hello, Cryptos!' )

resulting in

#=> "33eedea60b0662c66c289ceba71863a864cf84b00e10002ca1069bf58f9362d5"

And building up to blocks with proof-of-work, timestamps, and more:

require 'digest'

require 'pp' ## pp = pretty print

class Block
attr_reader :data
attr_reader :prev
attr_reader :difficulty
attr_reader :time
attr_reader :nonce # number used once - lucky (mining) lo

Blockchain vs (Hyper) Ledger - Inside the ledger-lite library / gem - Add Transactions One Block at a Time and Balance the Accounts (Books) w/ Ruby

Hello,

yesterday's Hyperledger talk notes titled Blockchain vs (Hyper) Ledger
-- Inside (Hyper) Ledger Lite - Add Transactions One Block at a Time
and Balance the Accounts (Books) with Ruby [1][2].