Tuesday, September 2, 2014

A quick way to load the baseball DB


I found the quickest way to load and reload the baseball database I've converted was to do two things:
  1. Cut the db into individual files
  2. Load those files one by one along with some logging
This is all accomplished through good, old-fashioned and very fast command line calls to SQLCMD .

There's a couple of great things about this script.

  1. It runs fast, even in a VM.
  2. It uses integrated security - no need to jump through hoops to make it work.
  3. The output is directed to a very specific file, which is rebuilt with every command.


The downsides of this approach aren't really "cons" as in "pros vs. cons", but more like limitations. For example:

  1. The file output of SQLCMD is overwritten every time. In command line mode, there is no SQLCMD flag to indicate I'd like the file "appended to" instead.
  2. The output file can be appended through concatenation, though, which, while a separate command line call, is still pretty simple.


Here's the script. I've saved this in my VM as a file on the desktop named "Make Lahman.cmd"


cls
sqlcmd -S localhost -d lahman -b -I -i lahman2013_tables.sql -o tempout.txt
type tempout.txt > finalout.txt
sqlcmd -S localhost -d lahman -b -I -i "lahman master.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
sqlcmd -S localhost -d lahman -b -I -i "lahman Fielding.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
sqlcmd -S localhost -d lahman -b -I -i "lahman Batting.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
sqlcmd -S localhost -d lahman -b -I -i "Pitching.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
sqlcmd -S localhost -d lahman -b -I -i "Teams.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
sqlcmd -S localhost -d lahman -b -I -i "PitchingPost.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
sqlcmd -S localhost -d lahman -b -I -i "lahman ManagersHalf.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
sqlcmd -S localhost -d lahman -b -I -i "lahman AllstarFull.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
sqlcmd -S localhost -d lahman -b -I -i "TeamsHalf.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
sqlcmd -S localhost -d lahman -b -I -i "TeamsFranchises.sql"  -o tempout.txt
type tempout.txt >>finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "Salaries.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "Schools.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "SchoolsPlayers.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "SeriesPost.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "lahman Managers.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "lahman HallOfFame.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "llahman FieldingPost.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "lahman BattingPost.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "lahman AwardsSharePlayers.sql"  -o tempout.txt
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "lahman AwardsShareManagers.sql"
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "lahman AwardsPlayers.sql"
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "lahman AwardsManagers.sql"
type tempout.txt >> finalout.txt
 sqlcmd -S localhost -d lahman -b -I -i "lahman Appearances.sql"
type tempout.txt >> finalout.txt


No comments:

Post a Comment