Describe when someone would need this information. For example "when connecting to wi-fi for the first time".
For VirtualBox Users
If working on a VirtualBox Machine, you may have notice that you can not change the resolution of your screen. Use the following instructions to fix this issue.
sudo apt-get update sudo apt-get upgrade sudo apt-get install dkms sudo apt-get install build-essential |
While in your VirtualBox Machine, bring your mouse to the top of the screen and choose devices
Choose Insert Guest Additions CD Image
A dialog Box will appear. Click: Run, then enter your password
When prompted, click Enter/return on your keyboard
Now restart your VirtuaBox machine
Your screen resolution should now be fixed. If not rerun all the steps above one more time.
When your VirtualBox machine comes back up, right click on the disk icon sitting in your Dock, and choose eject.
Starting on your task
If you don't already have git installed on your computer, run the command: sudo apt-get install git
Go to the following URL: https://github.com/dmc-uilabs
Create clone of the dmcdb repository, restservices repository, and frontend repository(for yaml)
A Jira account should be created for you, and your GitHub Account should have Access to the dmc-uilabs repositories. If you do not have access, reach-out.
Create create a new branch in each of these related to your task.
As you work on your branch, be sure to perform a git pull on your branch to make sure it is up to date with newly merged code.
Installing everything for your restservices testing environment
java -version |
sudo apt-get update sudo apt-get upgrade sudo add-apt-repository ppa:openjdk-r/ppa sudo apt-get update sudo apt-get install openjdk-8-jdk sudo apt-get update close and reopen terminal sudo update-alternatives --config java |
ls /usr/lib/jvm/ |
cd ~ nano .bashrc when in terminal or nano .zshrc When is zshell |
Add the following new lines to the bottom of the file (those not referencing JAVA will be used later)
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
Save your changes
Close and reopen the terminal
Run the command: echo $JAVA_HOME
You should get back the location you added to the .bashrc file
Installing Tomcat
If you have not installed Java yet, install it first before installing tomcat
sudo apt-get install tomcat7 sudo apt-get update |
Close and reopen terminal, then run:
sudo service tomcat7 start |
If you get an error along the lines of set JAVA_HOME or no JDK installed, do the following:
sudo nano /etc/default/tomcat7 |
In this file where you see #JAVA_HOME, erase the entire line and replace it with:
JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
Close and reopen the terminal, then re-run the command the command to start tomcat7
Wait a few moments, and you should get an [OK] back
Go to a browser and go to the following URL: localhost:8080. This should say It works!
Now you can run: sudo service tomcat7 stop |
sudo apt-get install pgadmin3 |
sudo apt-get install maven |
If you have issues running maven or mvn, the following commands can at time be useful.
mvn clean mvn dependency:resolve |
If working with a 64 bit linux machine, the following URL will take you to a location from which you can download this IDE:
Once downloaded, go to your downloads folder, right click the newly downloaded eclipse folder, then choose extract here.
Change directories to the new eclipse folder that was just created.
Open eclipse application
When selecting a workspace, click OK to set it as default
After the application is finished loading, click on Workbench in the top right-hand corner
Next click, Window → Show View → Project Explorer
This lets you view your project structure on the left hand side.
In order to set up postgres to run locally, you will need to first have a database installed on your local machine
If you have not already done so, got to the following URL and close the dmcdb repository: https://github.com/dmc-uilabs
In a terminal, navigate to the dmcdb folder.
If you already had the repository downloaded, make sure to do a git pull before proceeding.
Configuring a local PostSQL Database:
The following instructions will set up a local Postgres database for developers to test REST services against.
All of these commands are done on the command line. Lines with # are comments, lines with $ are shell commands, and lines with no prefix are postgres shell commands.
$ sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list' $ wget -q https://www.postgresql.org/media/keys/ACCC4CF8.asc -O - | sudo apt-key add - $ sudo apt-get update $ sudo apt-get install postgresql-9.3 postgresql-contrib-9.3 |
$ psql --version |
$ sudo -u postgres psql postgres |
\password postgres |
CREATE USER gforge WITH PASSWORD 'gforge'; |
\du |
\q |
$ sudo -u postgres createdb -O gforge gforge |
$ sudo -u postgres psql -c "\l" |
$ sudo -u postgres psql postgres |
\connect gforge |
\q $ sudo nano /etc/postgresql/{your postgres version goes here}/main/pg_hba.conf |
#In this file, search for 'postgres'. You should find a line that has the following:
Local | All | Postgres | Peer |
#In this file, change the word 'peer' to 'md5'. Save the changes and navigate to the folder with the stable version of the dmcdb
Flyway: The DMC uses Flyway, which is a database migration program used for tracking changes to the database. Every time an update, change, or a new file is made in the database flayway is used to keep track of these individual changes, making it easy to revert changes made by developers.
Once you have configured your PostSQL Database run the following commands to start a new database instance:
sudo -u postgres psql -c "DROP DATABASE gforge" |
sudo service postgresql restart |
psql -U postgres -c "CREATE DATABASE gforge WITH OWNER gforge;" |
sudo -u postgres psql -c "\l" |
./flyway clean migrate info -configFile=conf/core/flyway.conf -url=jdbc:postgresql://localhost:5432/gforge -user=gforge -password=gforge |
./flyway migrate info -configFile=conf/data/flyway.conf -url=jdbc:postgresql://localhost:5432/gforge -user=gforge -password=gforge -locations=filesystem:./sql/data/dev2 |
The database is now running and can be viewed by pgAdmin and can be used to run restservices locally.
sudo apt-get update cd ~/DMC/frontend curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.2/install.sh | bash //if not using zshell, only run .bashrc source ~/.bashrc source ~/.zshrc nvm install v5.1.1 //Check to see that your node version is 5.1.1 node -v //If you have issue running the commands below, you may need to close all your terminal windows, return to your frontend folder and try them once more. sudo apt-get install npm npm install npm install nodejs npm install -g bower bower install npm install gulp npm install babel-register sudo ./post-install.sh //Using Yarn to Serve and Build Frontend curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list sudo apt-get update && sudo apt-get install yarn //Download the following file and un zip in the root of frontend directory https://s3.amazonaws.com/dmc-build-aritifacts/0.1.5/bower_components.tar.gz //Serving Front end //in one terminal tab run the following in the following directory: /dmcfront/jsonserver node server.js //in another tab run the following in the root of the frontend directory gulp serve //building the frontend for testing //if you already have a dist folder in the root of your frontend directory to remove the dist folder rm -rf /dist //To build the dist folder, run the following in the root of the frontend directory: gulp build |
The following are the stages your task for the most part should go through
Checking the .yaml file
If you have not yet done so, clone the frontend repository from github.
Change directories to the following: frontend/restapi
Open the file called: DMCInterfaceSwagger.yaml
Check and make sure that the .yaml file has information about your endpoint already there.
If it is missing, you will need to update your endpoint to reflect your code after it is completed.
Working on the data base
Follow the instructions listed above for creating a local database and connect to the database using pgAdmin (make sure you are on your own branch).
Open the folder structure as such: Server Groups → Servers → <Local Database running with postgres> → Databases → gforge → Schemas → public → Tables
Locate the table(s) that will be required for your endpoint(s) to function
If your table(s) is missing or has missing or incorrect column names or types, or does not have existing sample data, you will need to do the following
You will need to generate the code required to create your table. To do this do the following
In pgadmin, right click the tables folder and choose new table.
Under properties, fill out the table name (table should use lowercase characters only)(the norm for table names with more than one word should look like this: multiple_word_table_name).
Under columns, click add to create a new column. Fill out the name field (do not use camel case here, follow the same convention as used in the step above) and chose a type for the field such as: text, date, integer, or serial (generally used for ids).
When creating an id column that should generate an id on its own, do the following
Create the column with a name and the type of serial.
Click on the constraints tab.
Make sure Primary Key is selected from the drop-down menu.
Click Add.
In the new window, click on the columns tab
In the column drop-down menu, choose the name of the column.
Find the table that you created in the tables folder in pgadmin.
Click on the table name and then click on the table icon (7th icon from the top left of the window).
This is a view of what your table will look like
At the bottom of the pgadmin window, there is a pane with code. Copy this code.
./flyway clean migrate info -configFile=conf/core/flyway.conf -url=jdbc:postgresql://localhost:5432/gforge -user=gforge -password=gforge |
./flyway info -configFile=conf/data/flyway.conf -url=jdbc:postgresql://localhost:5432/gforge -user=gforge -password=gforge -locations=filesystem:./sql/data/dev2 |
You will now update 3 files in the database. File names and locations are as follows:
dmcdb/sql/core/tables
Create a new table file.
Edit the file and paste the code that you copied in the step above here.
Cut the line that says CONSTRAINT table_name_PKEY PRIMARY KEY (id)
erase the lines: )WITH ( UIDS=FALSE
Save the file.
/dmcdb/sql/core/constraints/
Create a new primary_key_constraints file.
Add the following lines to the file:
- -Name: table_name_pkey; Type: CONSTRAINT; Schema: public; Owner: gforge; Tablespace: - ALTER TABLE ONLY <table_name> ADD <paste what you cut earlier here>; |
Save the file.
dmcdb/data/dev2/
Create a new sample data file.
Add three lines of sample data here. Use the following as an example for syntax
insert into table_name (column_name one, column_name_two) values ('text', 'textAlso');
Explanation:
table_name: use the name of your table here in its place.
The column names should be listed in order separated by commas.
The sample data must be listed in the same order as you list the column names.
Text data should be quoted using apostrophes and separated by commas. Numbers should not be quoted at all.
The id that should be auto generated should not be listed in the column name or the sample data.
Rerun postgres commands to drop database, create a database, check that the data base was created, add the tables to the database, and add the sample data.
Remember, you can not use the above commands if you are connected to the database with pgadmin, you will need to close pgAdmin, run the commands and then reopen pgAdmin..
Check to make sure you can open your table and that it has sample data in it. Also try to add another line of information. pgAdmin should handle generating the serial id, and the next line.
Once you've confirmed the information in the database is correct and updated you can now start working on your code.
Working on your code
Make sure that if you have not yet cloned the restservices repository from GitHub, that you do so now.
Also, make sure you create a new branch from the master branch.
Open Eclipse
Refresh Eclipse
Testing your code with swagger
Manual
Once you have code you would like to test, start drop and rebuild your local database. Next, fine the Config.java file located in the following directory: restservices/src/main/java/org/dmc/services/ Change the first 6 lines to be the following: public static final String DB_PORT = "5432"; public static final String DB_IP = "127.0.0.1"; public static final String DB_HOST = "jdbc:postgresql://" + DB_IP; public static final String DB_NAME = "gforge"; public static final String DB_USER = "gforge"; public static final String DB_PASS = "gforge"; Save the changes in this file. Next, add the following to the bottom of your /etc/profile file by using the command:sudo nano /etc/profile Run the command: source /etc/profile export DBip="127.0.0.1" export DBport=5432 export DBuser="gforge" export DBpass="gforge" export SOLR_BASE_URL="http://<SOLR IP Address>/solr" export ActiveMQ_URL="<ActiveMQ IP Address>" export ActiveMQ_Port=61616 export ActiveMQ_User="active" export ActiveMQ_Password="active" Next update the following file restservices/src/main/java/org/dmc/services/Application.javaAll references to DB under the @Beam should be replaced with the following: ds.setUser(Config.DB_USER); ds.setPassword(Config.DB_PASS); ds.setServerName(Config.DB_IP); ds.setPortNumber(Integer.parseInt(Config.DB_PORT)); ds.setDatabaseName(Config.DB_NAME); Finally on debian operating systems update the file /etc/default/tomcat7 (also add this to the /etc/tomcat7/tomcat.conf file as well) Add the following to the bottom of the file: DBip="127.0.0.1" DBport=5432 DBuser="gforge" DBpass="gforge" SOLR_BASE_URL="http://<SOLR IP Address>/solr" ActiveMQ_URL="A<ctiveMQ IP Address>" ActiveMQ_Port=61616 ActiveMQ_User="active" Note that the above system environments do include IP addresses from live SOLR and ActiveMQ machines. If the above addresses do not work you will need to get these IP address from a DMC developer Next, go to the restservices directory. Run the command: sudo service tomcat stop Next, run the command: mvn package -P swagger When the .war file build is complete, go to the following directory: /var/lib/tomcat7/webapps If there are any files in the webapps folder, delete them. The command to do this is: sudo -r * Next, go to the following directory restservices/target/ Rename the file named: dmc-site-services-0.1.0-swagger.war to rest.war; you can use the following command to do so: mv dmc-site-services-0.1.0-swagger.war rest.war Move the rest.war file to the /var/lib/tomcat7/webapps/ folder using the following command: sudo mv rest.war /var/lib/tomcat7/webapps/ Now start tomcat using the following command: sudo service tomcat7 start Go to the following URL in a browser: localhost:8080/rest/services/2 or http://localhost:8080/rest/swagger-ui.html. This may take some time to load. Wait a few moments. |
Automatic
Once you have code you would like to test, start drop and rebuild your local database.
Next, fine the Config.java file located in the following directory.
restservices/src/main/java/org/dmc/services/
Change the first 6 lines to be the following.
public static final String DB_PORT = "5432"; public static final String DB_IP = "127.0.0.1"; public static final String DB_HOST = "jdbc:postgresql://" + DB_IP; public static final String DB_NAME = "gforge"; public static final String DB_USER = "gforge"; public static final String DB_PASS = "gforge"; |
Save the changes in this file.
Run the command: source /etc/profile
export DBip="127.0.0.1" export DBport=5432 export DBuser="gforge" export DBpass="gforge" export SOLR_BASE_URL="http://<SOLR IP Address>/solr" export ActiveMQ_URL="<ActiveMQ IP Address>" export ActiveMQ_Port=61616 export ActiveMQ_User="active" export ActiveMQ_Password="active" |
ds.setUser(Config.DB_USER); ds.setPassword(Config.DB_PASS); ds.setServerName(Config.DB_IP); ds.setPortNumber(Integer.parseInt(Config.DB_PORT)); ds.setDatabaseName(Config.DB_NAME); |
Save the changes in this file.
Run the command: sudo service tomcat7 stop
Go to the following directory:
/var/lib/tomcat7/webapps
If there are any files in the webapps folder, delete them. The command to do this is:
sudo -r *
Next, go to the restservices directory.
Run the command: mvn spring-boot:run -P swagger
Compilation now happens
When completed, open a web browser and got to the following URL:
You will need to then write tests for the code you have written.
Once you believe your tests are complete, you can test them by first closing all terminals you have.
Open a new terminal.
Make sure postgres is running an instance of your database.
Change directories to your rest services folder
Then write the following command
mvn -Dtest=<TestName> test
DO NOT include the triangle brackets in your command
Make sure your environment variable are set or are included in your Config.java file so that your local database is being referenced.
Submission/Code Review (for database changes and resetservice changes)
To submit your cod for code review use the following commands:
Git status
Shows you what files you have made changes to in the repository
git add -A
-A add all files that have changes, by giving the full file path in its place you can add specific files
Git status
this checks which files you've added to the staging area.
Git commit -m “information about commit”
-m lets you add information about the commit.
“” are used for capturing comments.
Git push origin BranchName
In place of branch name include your actual branch name.
This will push your changes to GitHub.
Go to GitHub, and navigate to your branch
Make sure to sync your branch with the master branch
Make a pull request and add a few reviewers.
Updating the .yaml file
After your changes have been merged, find the .yaml file mentioned in the steps above and edit it as well. Make sure to add tags live and has tests to your endpoint. Submit this for code review.
Documentation on this is coming soon.
#!/bin/bash echo "Changing Directories" cd dmcdb echo "Droping the current database" sudo -u postgres psql -c "DROP DATABASE gforge" echo "creating a new database" sudo -u postgres createdb -O gforge gforge echo "Confirm new database (gforge) was created" sudo -u postgres psql -c "\l" echo "creating tables" ./flyway clean migrate info -configFile=conf/core/flyway.conf -url=jdbc:postgresql://localhost:5432/gforge -user=gforge -password=gforge echo "filling tables with sample data" ./flyway info -configFile=conf/data/flyway.conf -url=jdbc:postgresql://localhost:5432/gforge -user=gforge -password=gforge -locations=filesystem:./sql/data/dev2 ./flyway migrate info -configFile=conf/data/flyway.conf -url=jdbc:postgresql://localhost:5432/gforge -user=gforge -password=gforge -locations=filesystem:./sql/data/dev2 echo "-----------------open pgadmin-----------------------" |
If restoring a database the following command is used
psql -U gforge gforge < azurePortal20170523.sql |
You can use the commands below to stop postgres from running and subsequently disconnect application that are accessing your database.
sudo service postgresql stop sudo service postgresql start |
#!/bin/bash echo "Stopping Tomcat." sudo service tomcat7 stop echo " " sleep 2 echo "Moving to restservices directory." echo " " cd ~/Desktop/dmc/restservices echo "Starting mvn spring-boot:run -P swagger" echo " " sleep 3 firefox localhost:8080/swagger-ui.html echo "When complete go to localhost:8080/swagger-ui.html in a browser." mvn spring-boot:run -P swagger |