Venkat$ echo

I'm a hard-core ORACLE programmer and lately I had to work with SQL server for a specific project. First thing I noticed is that its a bit hard to search for column names, table_names (*across all databases*) etc in sql server. In oracle, we have the DBA views like DBA_tables, DBA_tab_columns etc, which are very handy for these purposes. In SQL sever we have a stored proc called "sp_msforeachdb" to somewhat do this task. But it opens up so many result windows and also creates one window for each DB, even if the DB doesnt have any results. So I ended up making few stored proc which will work somewhat similar to the oracle's DBA views. Use the below script to create the working tables. create table my_dba_tables( db sysname, table_name sysname, object_id int); create table my_dba_tab_columns( db sysname, table_name sysname, object_id int, table_column sysname); create table my_dba_objects( db sysname, object_name sysname, object_id int); Download the stored...

I'm doing an IOS app which uses Parse's(www.parse.com) Cloud code. Being a IOS developer I found it a bit tedious to test the output of my Cloud Code(CC) using objective C (especially when I'm changing and developing the cloud code) I thought it would be easy if I can check the output of my CC using a JavaScript directly from the browser, because you can see the output of you CC immediately and also get a better understanding of what values are being returned. so I made a quick template which uses Parse JS API to   Login to Parse( In my app you can call a CC only if you login as a App user) Call could code and display its result. Download the template from this link - Venkat-GitHub-Page-Parse Remember to edit the index.html file and put your <app_ID> and <java-script-key> at Parse.initialize Edit the login details Edit the Cloud code function details and parameters( if any)

This example is to show that you can draw your own shapes in tableau ( not just on maps) and use that for operational purposes. Lately I have been working on hadoop and big-data and want to monitor which servers are up and running, so that it is easy for operational monitoring purposes. In the below example I have combined the shapes and the server status( "Running") on the same file, but if you cant/dont want to do that you can use the data blending technique to combine 2 different data source You can download the source from the below "Tableau Public" link Learn About Tableau

Click here for other Tableau related Blogs Requirement is to convert data in ESRI Map shapes into Tableau format. Tools Required Quantum GIS -  http://www.qgis.org/en/site/ Python  Download the below Python Code -  https://github.com/venkat-vs-id/python import csv import re import sys #def convertESRI2TableauFormat( p_inputfile, p_outputfile): l_outfileName = 'c:/ESRI_suburb_tab_format.csv' # make changes here.. l_ESRIFileName ='c:/ESRI_suburb.csv' # make changes here.. rdx=0 csv.field_size_limit(sys.maxsize) outfile = open( l_outfileName, 'w', newline='') csvWriter = csv.writer( outfile, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL) with open( l_ESRIFileName, 'r') as csvfile: csvR = csv.reader( csvfile, delimiter=',', quotechar='"') for row in csvR: rdx += 1 ci=0 rowValues = [] rowValues.append(row[1]);# ssc_code rowValues.append(...

Click here for other Tableau related Blogs Please note that TABLEAU does not advice anyone to access their DB. So if you are doing it you are on your own. Tableau has 2 DB Postgresql for managing all its reporting repositories FireBird for mapping repositories Tools Required pgAdmin 3 for PostgreSql Flamerobin for FireDB Accessing Tableau's PostgreSQL DB Tableau has 2 buitin userid tableau = with read-only access to some of Tableau's view tblwgadmin = admin user will all privileges Please note tblwgadmin is like a admin user, so if you are using this you have to be very careful. Or if you dont admin privs but just need enough access to read the tables then use just "tableau" user. Get access for "tableau" user.  First set a password for "tableau" userid open command prompt go to <Tableau_Home>\Tableau Server\7.0\bin run -  tabadmin dbpass <give a password> Restart the server Now open pgAdmin3 and from...

Click here for other Tableau related Blogs I had requirement to shapes we had in Oracle Spatial data( which we used for MapInfo) in Tableau. Since tableau doesnt support Oracle spatial type columns, I had to extract that data in Tableau readable format. Here are the steps to convert oracle spatial data into format useable by Tableau. Run the Oracle package ven_OraSpatial_2_TabData in your oracle environment. Now run the below "ConvertDataFormat" code to extract the data into Tableau format Now use extracted data in tableau ( refer to my blog -  Tableau Maps - Custom shapes / polygon - Basics ) You can also download the code from this link -   VEN_ORASPATIAL_2_TABDATA.zip Oracle package ven_OraSpatial_2_TabData SPEC CREATE OR REPLACE PACKAGE VSUBR.ven_OraSpatial_2_TabData as ----- TYPE TYPE rec_LatLong IS RECORD( Polygon_ID NUMBER, LAT NUMBER, LON NUMBER ); TYPE tab...

Recently I came across a puzzle which can be solved by using Bayes probability theorem.  Thought I will share that. Puzzle: If a person has malaria, there is a 90% chance that his/her test results are positive. However the test result are not very correct, There is a chance for 1% error. Also only 1% of the total population get affected by Malaria. Now one person's test result came out as Positive. Whats the odds that he will actually have Malaria. Bayes theorem: Before getting into the solution - here is the simple explanation of Bayes theorem. Bayes theorem gives you the actual probability of an event from the measured test probabilities and the skewness . In other words - If you know the real probabilities and the chance of skewness ( i.e. false positive and false negative), you can make correction for measurement errors. Solution: Applying Bayes formula                             ...

Click here for other Tableau related Blogs In many situations we would like to draw our own custom shapes on the maps, so that we can report the data more geographically. To draw custom shapes in Tableau, you have to first understand how the custom shapes works within tableau. Image-1 : Polygon shapes in Tableau 1. Each shape is a polygon. 2. Each point_id links to a co-ordinate( i.e lattitude,longitude). 3. The point_id value is used to tell tableau the order (or path) in which the polygon has to be drawn. 4. So in the above example SatisticalArea = "area-1" has more than 1 shape and see how the polygon_IDs are different and for each polygon_ID the point_id are in a sequential order. This how a sample data looks like Attached is the result from tableau (just has custom shape of few areas) Here is an working Example - Ven_Custom_Shapes_Example

Sometimes you like to get the user's attention to a specific control like button or image etc. In Mac I have seen the icons jumping of the "Dock" to get our attention, So I thought why not try a similar behaviour inside a App. So below is the sample app where I have made a button to jump. IOS_JumpingButton_Example_Source_From_gitHub

Click here for other Tableau related Blogs In many report we might have requirements to restrict a "filter values" based on value selected on a another "filter". Usually we used to use AJAX techniques to achieve this. In tableau its quite easy to achieve the same result, by just checking few options. Here is a working sample -  ven_AJAX_Like_FilterSample Lets assume you have 3 filters continent, country  and city. Make a "quick Filter" for all three of them Optional Step: Change the filter to a multi-drop down box or just leave it as is Now from the "quick filter" options ( the down arrow on the right corner of quick filter) of the Country filter  and check "only relavant values". Do the same for City filter Now you will see that the country filter values and city filter values gets adjusted based on what value you select on other filters.

This post will link to all the other posts on Tableau. Ajax Filters .( i.e when you select one filter it should restrict the values on other filters based on the value selected.) Tableau Maps - Custom shapes / polygon - Basics. Convert Oracle Spatial Data into Tableau Format Tableau - Accessing Tableau's DB Tableau : Convert ESRI shapes into Tableau Format My CV - with Story Board

"Word Count" I know this is one of the common example you will find when searching for hadoop examples. The code for this comes along with the hadoop installation.  This is a very very simple example which you can use to understand how the hadoop code works. Steps: Make some sample files. I have made 2 files - you can download them from this link - test sample files Now load these files into Hadoop's HDFS ./hadoop-1.0.4/bin/hadoop dfs -copyFromLocal /home/venkat/Documents/*.txt /source_data/ You can also see the uploaded files using the hadoop web portal. As i said earlier hadoop installation should contain an example jar which has got "word count" as one of the example. Here is where you can find that example jar /hadoop-examples-1.0.4.jar Use this command to see the classes related to "word count" jar -tvf ./hadoop-1.0.4/hadoop-examples-1.0.4.jar | grep 'wordcount.class' -i If you would like to see the source...

Hadoop shell commands are very similar to linux shell commands. The Below links gives the complete set of commands Hadoop Shell Commands  - http://hadoop.apache.org/docs/r0.18.3/hdfs_shell.html Below are some of the important ones are ./hadoop-1.0.4/bin/hadoop dfs -ls / ./hadoop-1.0.4/bin/hadoop dfs -mkdir /source_data ./hadoop-1.0.4/bin/hadoop dfs -lsr / ./hadoop-1.0.4/bin/hadoop dfs -mkdir /tmp/tmp1 ./hadoop-1.0.4/bin/hadoop dfs -copyFromLocal /home/venkat/Documents/*.txt /source_data/ ./hadoop-1.0.4/bin/hadoop dfs -rmr /source_data/*.txt

Hadoop 1.0.4 on Ubuntu Linux 12.04 ( Single Node )- Part -3/3 Before starting the hadoop, you need to format the HDFS (i.e the hadoop filesystem) via the namenode. What this does is basically clears the directory values given in { dfs.name.dir}. This is the directory used by name-node to keep details of all the data in the data-node( i.e files), basically every detail required to track the which data sits in which data-node. Please understand that it does not store any data itself, it just stored the details of which data sits on which data-nodes. use the below command to format the name-node. ./hadoop-1.0.4/bin/hadoop namenode -format and the result looks like below. Now you can start your hadoop. ./hadoop-1.0.4/bin/start-all.sh and there result looks like this. Also check if all the required hadoop processes are running by typing jps.   If you face any problems then have a look into the logs file and investigate. and If you want to stop hadoop use teh bel...

Hadoop 1.0.4 on Ubuntu Linux 12.04 ( Single Node )- Part -2/3 Download Hadoop Download a stable hadoop version from  http://hadoop.apache.org/ I downloaded the this version - hadoop 1.0.4 and this is the link to it stable hadoop download page . Download the file (hadoop-1.0.4.tar.gz) and copy it under the home directory of hadoop_usr ( ie. /home/hadoop_usr/) Steps to Install Login to terminal as hadoop_usr and extract the contents of the gz file tar -xvf hadoop-1.0.4.tar.gz We have to edit the following configuration file $HADOOP_HOME/conf/hadoop-env.sh $HADOOP_HOME/conf/hdfs-site.xml $HADOOP_HOME/conf/core-site.xml $HADOOP_HOME/conf/mapred-site.xml $HADOOP_HOME/conf/masters $HADOOP_HOME/conf/slaves $HADOOP_HOME/conf/hadoop-env.sh set the Java home # The java implementation to use.Required. # export JAVA_HOME=/usr/lib/j2sdk1.5-sun export JAVA_HOME=/usr/lib/jvm/java-7-oracle $HADOOP_HOME/conf/hdfs-site.xml before editing this file create the f...

Hadoop 1.0.4 on Ubuntu Linux 12.04 ( Single Node )- Part -1/3 There are few ways to install Hadoop.  Cloudera Distibution Hadoop PPA Stable download from hadoop.org I prefer to use the download from hadoop.org. One of the main reason to select this is whenever there is a new version of hadoop, I need not wait for someone else to release their version including the newer version.  Should we not be installing hadoop on multi nodes? Yes. But my objective is to set-up a hadoop environment on my laptop, so that I can play around and also get a better understanding of Map-Reduce. For that single-node set-up is sufficient. Prerequisites Make sure you have already installed Java 1.6 on my machine. if not please follow this link -  Install Java 1.6 Create a separate user & usergroup for hadoop. I like to keep a dedicated user for hadoop. It is much easier when it comes to giving permissions and for various admin acitivies. It is ...

Hadoop Setup and Architecture Hadoop | Installation ( SIngle-Node ) - 1/3 Hadoop | Installation ( SIngle-Node ) - 2/3 Hadoop | Installation ( SIngle-Node ) - 3/3 Hadoop File System Commands Run a Hadoop Example - Word Count

Install Apache Maven 3  on Ubuntu 12.10 Follow the below steps to install Maven3.  Maven is a important tool which we might require to build some of the apache projects. So lets install it. Open you your terminal and type in the following commands. sudo apt-get install maven If you get an error message like  E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem. then run the below command sudo dpkg --configure -a After running the above command, again run the first command sudo apt-get install maven once the installation is over, then check the version to make sure if the installation was successful. mvn -version

Install Oracle(Sun) Java 1.7.x on Ubuntu 12.10 My windows Visa laptop crashed recently and I decided to get rid of windows Vista ( enough is enough) and install Linux on my laptop. So did a fresh install of Ubuntu 12.10 on my old laptop and had to install all required software on it. So I thought why not blog it. Also I'm planning to install Hadoop-Hbase-Mahout on this machine and the first step to hadoop and others is installing JAVA. So here are the steps to install JAVA on Ubuntu 12.10 Open you your terminal and type in the following commands. sudo add-apt-repository ppa:webupd8team/java sudo apt-get update sudo apt-get install oracle-java7-installer once the installation is over, then check the version of the JAVA installed to make sure if the installation was successful. java -version