Math 511: Linear Algebra
Underdetermined and Overdetermined Systems
Answer all of the questions in order and in a single pdf document. If you create a graph, include it with your pdf along with your responses. Do not create your plots with pen and paper. You must create your plots using your favorite graphing software. I suggest 
GeoGebra$\textregistered$, or
Desmos$\textregistered$.
Exercise 1: Sensor Calibration 
A K-type thermocouple produces a small voltage that depends approximately linearly on temperature. To calibrate the sensor, an engineer immerses the thermocouple in three temperature baths whose true temperatures are known (measured with a reference instrument). She records the voltage at each bath:
| True Temperature | Measured Voltage |
|---|---|
| $T(^{\circ}C)$ | $V\,(mV)$ |
| 25 | 1.0 |
| 25 | 2.1 |
| 100 | 4.0 |
She models the relationship as
$$ V = aT + b $$
where $a(mV/^{\circ}C)$ is the sensors' sensitivity and $b(mV)$ is its offset at $0^{\circ}C$.
(a) Write down the linear system for $a$ and $b$.
(b) What is the size of this linear system? Is it square, overdetermined, or undetermined?
(c) Use the first two equations to solve for $a$ and $b$. Then check whether the resulting line $V = aT+b$ correctly predicts the third measurement at $T=100^{\circ}C$? What do you conclude?
(d) Repeat part (c) using the last two equations to solve for $b$ and $c$. The check the prediction at $T=25^{\circ}C$. Do you get the same line?
(e) Explain in your own words why this linear system has no solution. What is physically happening in the calibration data?
This is a typical situation in engineering and data science: more measurements than unknowns, and no exact solution. Later in the course we will study least squares, a technique that finds the line $V = aT + b$ that comes closest to all three measurements — the line that minimizes the total prediction error even though no line satisfies all the data exactly.
Exercise 2: Production Planning
A small electronics shop assembles three products from two limited resources each day: technician time (hours) and soldering-station time (hours). Each unit requires:
| Sensor module | Controller board | Display panel | |
|---|---|---|---|
| Technician (hr) | 2 | 3 | 1 |
| Soldering (hr) | 1 | 1 | 2 |
Let $x_1$, $x_2$, and $x_3$ denote the number of sensor modules, controller boards, and display panels produced in a day. The shop has 24 technician-hours and 16 soldering-hours available each day, and the manager wants to use all of both resources (no idle staff, no idle equipment).
(a) Write down the linear system for $x_1$, $x_2$, and $x_3$.
(b) What is the size of this linear system? Is it square, overdetermined, or underdetermined?
(c) Eliminate $x_1$ between the two equations. Solve for $x_2$ in terms of $x_3$, and then for $x_1$ in terms of $x_3$. Describe the set of all solutions.
(d) A production plan is physically meaningful only if every $x_i \geq 0$, $1\le i\le3$. For which values of $x_3$ does the solution from part (c) satisfy $x_1 \ge 0$, $x_2 \ge 0$, and $x_3 \ge 0$? Give one physically meaningful example.
This is a typical situation in operations and economics: more variables than constraints produces a family of feasible plans, and the manager must impose additional considerations — profit, demand, inventory limits — to pick a single plan. Later in the course we will see techniques for representing and choosing among one-parameter families of solutions systematically.
Exercise 3: Linear Model
A data scientist at a university wants to predict a student's end-of-semester score from three features she can measure during the semester: weekly hours studied, attendance rate, and prior GPA. She proposes the linear model
$$ s = a\cdot h + b\cdot r + c\cdot g $$
where $s$ is the predicted score, $h$ is weekly hours studied, $r$ is attendance rate (between $0$ and $1$), $g$ is prior GPA (between $0$ and $4$), and $a$, $b$, $c$ are the coefficients she needs to learn from data.
She pulls records for three past students whose final scores are known:
| Student | Hours studied $h$ | Attendance $r$ | Prior GPA $g$ | Score $s$ |
|---|---|---|---|---|
| A | 10 | 0.8 | 3.0 | 82 |
| B | 5 | 1.0 | 2.0 | 61 |
| C | 15 | 0.5 | 4.0 | 98 |
(a) Write down the linear system for $a$, $b$, and $c$.
(b) What is the size of this linear system? Is it square, overdetermined, or underdetermined?
(c) Solve the linear system using Gaussian elimination.
(d) Use the fitted model to predict the end-of-semester score of a new student who studies $8$ hours per week, has an attendance rate of $0.9$, and a prior GPA of $3.5$.
This is what "fitting a linear model" literally means in data science and machine learning. Each row of the data table contributes one equation, each feature contributes one unknown coefficient, and solving the linear system produces the model. In a real application, the data scientist would use far more than three students, producing an overdetermined system that is solved approximately by least squares — a technique we will study later. The mathematics is the same; only the number of rows changes.
Notice that the fitted coefficient on prior GPA is negative. This is a warning about fitting a model to very little data: with only three students, any pattern the system happens to capture is treated as truth, even if it contradicts what we would expect physically. Real-world data scientists use far more data and additional techniques to avoid this.
Exercise 4: Provide an Example
Answer Yes/No - and Explain.¶
In each of the questions below answer Yes/No. If the answer is Yes, create an example of the specified linear system. You may create a graph to illustrate your response, however it is not required. If the answer is No, then explain in your own words why there is no linear system with the specified properties?
1. Are there consistent, underdetermined linear systems with three independent variables?¶
2. Are there consistent, overdetermined linear systems with three independent variables?¶
3. Are there inconsistent, underdetermined linear systems with three independent variables?¶
4. Are there inconsistent, overdetermined linear systems with three independent variables?¶
Exercise 5: Short Answer
Short Answer¶
1. Explain why you would expect an overdetermined linear system to be inconsistent.¶
2. Explain why you would expect an underdetermined consistent linear system to have infinitely many solutions rather than a unique one.¶
3. A student measures the temperature in a chemistry lab using three different thermometers placed in the same beaker of water.¶
The student wants to find the "true" temperature by treating each reading as an equation $T = m_i$, where $m_i$ is the reading of thermometer $i$. Is the linear system square, overdetermined, or underdetermined? Why is it almost certainly inconsistent? What does this tell one about a plan to find a single "true" temperature?
4. A traffic engineer monitors flow through a four-way intersection.¶
The engineer writes down a conservation equation at each of the four corners (flow in equals flow out). The resulting system has four equations and several unknown flow rates on the connecting streets. Surprisingly, even though there are four equations, the system is still underdetermined. Give a plausible explanation for why the four conservation equations do not act as four independent constraints.
