Programming Fundamentals

Learning Objectives

Following this assignment students should be able to:

Use local and global macros to store values and variable lists

Store and reuse results from commands using macros and scalars

Write custom subroutines using capture program drop and program define

Automate repetitive tasks using forvalues and foreach loops

Apply conditional logic (if, continue) inside loops to control code flow

Generate multiple summaries, regressions, and graphs with minimal repeated code

Reading

Topics

Macros
Storing data
Loops
Writing commands

Readings

Reading: Programming

Lecture Notes

Exercises

Using Locals in Regressions (10 pts)

In this exercise you will use local macros to store variable lists and reuse them in multiple regressions. For this week’s assignment we will use eth_allrounds_final.dta so start by loading that data set.

Create a local macro named controls that contains three plot- or household-level control variables:
- crop_shock (1 = experienced shock, 0 = no shock)
- irrigated (1 = irrigated, 0 = not irrigated)
- hh_asset_index (household asset index)
1. Use this macro to regress of yield_kg on nitrogen_kg and the controls stored in controls but only if the variable crop_name equals maize. What are the coefficients on each control that is statistically significant at the 90% level?
2. Now modify the macro definition so that the controls also include female_manager (1 = if the plot manager is female, 0 = if male). Re-run the regression without changing the regression lines themselves. What are the coefficients on each control that is statistically significant at the 90% level?
Using Locals to Store Results (10 pts)

Using eth_allrounds_final.dta,
1. Summarize yield_kg and store the mean and standard deviation from this command in locals named mean_yield and sd_yield using r(mean) and r(sd). Use these locals to create a standardized yield variable called yield_std. Recall from your stats classes, to standardize a variable you subtract the mean value from the variable and then divide the difference by the standard deviation. What is the mean and standard deviation of yield_std?
2. Use the display command and your locals to print the following sentence “Mean yield on Ethiopian plots is `mean_yield' kg with standard deviation of `sd_yield' kg.”
Using Globals (10 pts)

In this exercise you will practice using a global macro to store a cutoff that is reused in several commands. Remember: in this course, locals are preferred almost always; this exercise is to help you understand how globals work and why they can be risky.

Define a global macro named lg_cut that stores the size (in hectares) above which you will consider a plot “large”. Use 1 hectare as the cutoff.

1. Create a new indicator variable large_plot that equals 1 if plot_area_GPS is strictly greater than $lg_cut and 0 otherwise. Label the variable as “= 1 if plot area > 1 ha”. How many large plots are there in the data set?

2. Use sum and if to compute the mean yield_kg on large and non-large plots. What is mean yield in each group?

3. Download the following code files and place this in the folder where you keep all your code. That should be the path that $code points to.
Add the line do "$code/00_main.do" to your code under **## 3.3. Run 00_main.do. What cutoff does the output claim was used (in the printed header and in the variable label)? What cutoff was actually used to generate large_plot?

4. Diagnose the inconsistency by adding these lines to the end of 03_tables.do and re-running 00_master.do
```
   gen         large_plot_using_current_global = plot_area_GPS > $plot_cutoff_ha
   tab         large_plot large_plot_using_current_global
```
Do large_plot and large_plot_using_current_global match? Why or why not?

5. Fix the problem by changing the name of the global in 02_revise. Re-run 00_main. What cutoff does the output claim was used?
Storing Results as Numbers (10 pts)

Here you will practice storing regression output in scalars and using them later.
1. Run a regression of yield (yield_kg) on fertilizer (nitrogen_kg) and plot characteristics (plot_area_GPS irrigated). Using the e() objects, store the R-squared and the number of observations from this regression in scalars named rsq_yield and N_yield. Using display, what are the R-squared value and number of observations?
2. Immediately after, run the same regression but with harvest_value_USD as the dependent variable instead of yield_kg This will overwrite e(r2) and e(N). Store the R-squared from the second regression in a local macro named rsq_harv. Using display, what is the R-squared value?
For Values (10 pts)

In this exercise you will use forvalues to loop over survey waves and compute summary statistics.
1. Use tab to determine how many survey waves (wave) are in the data set. How many waves are in the data set?
2. Use a forvalues loop to summarize yield_kg separately for each wave. Your code should:
  - Loop over all integer values of wave observed in the data (for example, 1/3 if there are 3 waves).
  - For each wave:
    - Display a header line like "Wave 1".
    - Run summarize yield_kg if wave == ....
  Example structure (modify the range to match your data):
```
forvalues w = 1/3 {
    display "------------------------"
    display "Summary for wave `w'"
    summarize yield_kg if wave == `w'
}
```
  What is mean yield in each wave?
3. Extend your loop so that for each wave it also summarizes nitrogen_kg. What is mean nitrogren use in each wave?
For Each (10 pts)

This exercise uses foreach to loop over lists of variables.
1. Use foreach to run a one-way tabulation (tab) for the following shock indicator variables:
  - crop_shock
  - pests_shock
  - rain_shock
  - drought_shock
  - flood_shock
How many plots experienced each type of shock?
1. Modify the loop so that it also calculates the share of plots that experienced the shock (i.e., where the shock variable == 1). For each shock, use sum, if, and r(mean) to store that share in a local macro named `share_`shock'` and display the result as:
```
* Print as percent with one decimal place
  display   "About " %4.1f (100*`share_`shock'') "% of plots experienced a `shock'."
```
What share of plots experienced each type of shock?
Combining Macros and Loops (10 pts)

Here you’ll combine local macros and foreach loops along with varlist to create new variables and run multiple regressions.
1. Create a local macro named logvars that contains three nonnegative variables that you expect to be skewed:
  - yield_kg
  - harvest_value_USD
  - totcons_USD
Use a foreach loop over the variables in logvars to create log-transformed versions of each variable:
- For each variable v in logvars, generate ln_v’ = ln(v')
- Label each new variable "log of v"
- Use sum to calculate the mean for each logged variable
What is the mean value of each variable?
1. Now define two more local macros:
  - local yvars ln_*
  - local controls plot_area_GPS irrigated nitrogen_kg female_manager
Use a foreach loop to run regressions of each outcome in yvars on the same set of controls. In each regression, what controls are not significant?
Conditional Loops (20 pts)

In this final exercise you’ll use programming if, foreach, and continue to control what happens inside a loop based on sample size.

Goal: For a list of variables, automatically summarize and graph them, but skip variables that have too few non-missing observations. Start bv defining a local macro named vars with the following variables:
- yield_kg
- harvest_value_USD
- nitrogen_kg
- totcons_USD
1. Write a foreach and varlist to loop over vars. Inside the loop, for each variable v:
  - Run `qui sum `v', detail
  - Store the number of non-missing observations in a local N using r(N) (remember to use = so that the local equals the expression and not the literal text r(N)). Then, still within the loop, have Stata display the number of observations for each variable. How many observations are there for each variable?
Check That Your Code Runs (10 pts)

Sometimes you think you’re code runs, but it only actually works because of something else you did previously. To make sure it actually runs you should save your work and then run it in a clean environment.

Follow these steps in to make sure your code really runs:
1. Restart Stata by closing the instance of Stata that you have open.
2. When you reopen Stata, type in the command line clear all. Or, better yet, put clear all at the top of your .do file, after the preamble but before any commands.
3. Rerun your entire homework assignment by clicking the Execute button to make sure it runs from start to finish and produces the expected results.
4. Make sure that you saved your code with the name of the assignment number (i.e., assignment_01). You should see the file in your git repo on your machine.
5. Make sure that your code will run on other computers (relevant for all assignments after the first assignment)
  - No cd. Use a project.do file instead to set directories
  - Use only relative paths in files other than the project.do file
  - Use / not \ for paths
Challenge 6 (10 pts)

This challenge builds immediately on the loop you wrote in exercise 8. The goal is to put everything together in something closer to real work. For a list of variables, we will write loops to produce:
- A table of summary statistics
- A histogram saved to disk for each variable
The loop you wrote for exercise 8 should look something like:
```
* create a local of continuous variables
    local				vars yield_kg harvest_value_USD nitrogen_kg totcons_USD

* loop over each variable and print out the number of observations	
	foreach v of varlist `vars' {
		qui sum 			`v', detail
		local				N = r(N)
		display				`N'
	}
```
Within this loop, add a programming if statement that for variables stricly less than (<) 63,450 displays as text the following: “Skipping `v' (only `N' non-missing obs)”. Recall that after a programming if ... statement you need {} and it is inside these brackets where you put the text you want displayes if the statement is true.

Within this if loop, us continue to to tell Stata to move on to the next variable (this will have the effect of Stata moving onto the next variable without graphing variables with observations < 63450).

Outside of the if loop, tell Stata what to do when the if statement is not true. That is, draw a hist of the variable `v'. Label the title "Distribution of `v'" and x-axis label "`v'". Save the graph with a name that depends on the variable, e.g. hist_yield_kg.png.

In the end, you should have one loop inside another. The outside loop is the foreach loop above. The inside loop is the one that follows if .... The code for the hist goes inside the first loop but not inside the second. That second loop tells Stata what to do when the if statement is true. If the statement is not true, than Stata will automatically draw the hist

Assignment submission checklist

Assignment

Learning Objectives

Reading

Topics

Readings

Lecture Notes

Exercises

Solutions