1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
|
# Module 4: Outline
1. Introduction to Data and Scientific Datasets
a. What is scientific data
b. Data Processing flow work
c. Intro to Pandas
d. Manipulating data frames
e. Problem 1: Create a dataframe from Numpy arrays
f. Problem 2: Selecting data from a dataframe to calculate work done.
2. Interpreting Data
a. Understanding your data
b. Purpose
c. Composition
d. Color
e. Problem 1: Composing or fixing a plot. Apply PCC
f. Data don't lie
g. Problem 2: Misleading plots by changing axis limits or omitting context. Explain *why* it's misleading.
3. Importing, Exporting and Managing Data
a. File types
b. Importing spreadsheets with pandas
c. Handling header, units and metadata
d. Writing and editing data in pandas
e. Problem: Importing time stamped data pressure and temperature data. Convert timestaps to datetime and plot timperature vs. time
f. Problem: Add metadata () [TBD]
4. Statistical Analysis I
a. Engineering Models
b. Statistics Review
c. Statistics function in python (Numpy and Pandas describe)
d. Statistical Distributions
e. Spectrocopy (basics)
f. Problem: Statistical tools in Spectroscopy readings (intensity vs wavelangth) to compute mean, variance and detect outliers.
g. Problem 2: Fit a Gaussian distribution to the same data and overlay it on the histogram.
5. Statistical Analysis II: Regression and Smoothing
a. Linear Square Regression and Line of Best Fit
b. Linear, Exponential and Power functions
d. Polynomial
e. Using scipy
f. How well did we do? (R and R^2)
g. Extrapolation and limitations
h. Moving averages
i. Problem 1: Fit a linear and polynomial model to stress-strain data. Compute R^2 and discuss which model fits better.
j. Problem 2: Apply a moving average to noisy temperature data and ocmpare ra vs. smoothed signals.
6. Data Filtering and Signal Processing
a. What is it and why it matters - noise vs. signal
b. Moving average and window functions
c. Frequency domain basics (sampling rate, Nyquist frequency)
d. Fourier transform overiew (numpy.fft, scipy.fft)
e. Low-pass and high-pass filters (scipy.singla.butter, filtfilt)
f. Example: Removing high-frequency noise from a displacement signal
g. Example: Removing noise from an image to help for further analysis (PIV)
h. Problem 1: Generate a synthetic signal (sum of two sine waves+random noise). Apply a moving average and FFT to show frequency components.)
i. Problem 2: Design a Butterworkth low-pass filter to isolate the funcamental frequency of a vibration signal (e.g. roating machinery). Plot before and after.
7. Data Visualization and Presentation
a. Review of PCC framework
b. Plotting with Pandas and Matplotlib
c. Subplots, twin axes, and annotations
d. Colomaps and figure aesthetics
e. Exporitn gplots for reports (DPI, figure size)
f. Creating dashboards or summary figures
g. Problem 1: Using pandas to plot spectroscopy data from raw data. Add labels, units, title, and annotations for peaks
h. Problem 2: Create a multi-panel figure showing raw data, fitted curve, and residuals. Format with consistent style, legend and color scheme for publication-ready quality.
|
