Loop for monthly or yearly averages by day number (in Julian day format)

Question

I have daily '.mat' files whose nomenclature is like 'Data_grid_day_ppt_ice003.mat', the last '003' is the date of the year in Julian day format, It can start from '001' and end in '365' and '366' depending on whether it is a leap year or not.

I need monthly averages of the data. Any idea how can I make loops for the date files?

Some dates can be missing also, so just counting the number of files would not work.

I am writing in Matlab, I was trying something like

year = linspace(2007,2016,10); % Years of data
months = linspace(01,12,12);% Months of the year
for n = 1:length(year) 
    foldery = int2str(year(n));
    if ~exist(folderyear) 
        continue
    else
        if mod(year,4) == 0 % if the year is divisible by four 
            dpm = [31 60 91 121 152 182 213 244 274 305 335 366];      
        else 
           dpm = [31 59 90 120 151 181 212 243 273 304 334 365];     
        end
        for j = 1:length(months) %loop over months
            if ~exist(months) 
                continue
            else
                for ii = 1:dpm(j)   %loop over days
                    % ... Not sure what to do here ...
                end 
            end
        end
    end
end

However, I am not able to work out the next step...

Answer 1

Firstly, don't use year as a variable name, because it is a built in Matlab function! Then be careful with your variable names, like confusing foldery and folderyear.

---

The easiest way to do this would be to just add the day of the year to the 1st Jan in that year, then get the month using Matlab's built in month function. This will take care of all of the leap year issues as long as you know the year (which you seem to). Also see this documentation, Date and Time Arithmetic, which confirms that adding a number to a datetime value will add full 24 hour days.

The key is this method for getting month from Julian day of year:

thisyear = 2017;               % Define which year you're in
J1 = datetime(2016,1,1);       % Set January first datetime 
dayofyear = 360;               % Christmas day as an example!
m = month(J1 + dayofyear - 1)  % Add day of year to Jan 1st, get month
% J1 + dayofyear - 1 = 25-Dec-2016 00:00:00
% m = 12
% month(J1 + dayofyear - 1, 'name') = 'December'

---

See this commented extension of your code for more detail...

y = 2007:2016; % Years of data [2007, 2008, ... 2016]
outputdata = cell(1,12);
for n = 1:length(y) 
    folderyear = int2str(y(n));
    if ~exist(folderyear, 'dir') 
        continue
    else
        % Get all .mat files within this year folder
        f = dir(fullfile(folderyear, '*.mat'));
        filenames = {f.name};
        % Set Jan 1st date
        J1 = datetime(y, 1, 1);
        % Loop over months
        for m = 1:12
            % Loop over files
            for myfile = 1:numel(filenames)
                % Get number from end of file name
                dayofyear = filenames{myfile};
                dayofyear = str2num(dayofyear(end-6:end-4));
                % Get month of day by adding to Jan 1st of that year
                % -1 used assuming Jan 1st is day 1. Not needed if Jan 1st is day 0.
                if month(J1 + dayofyear - 1) == m  
                    % hurrah, this file is in the month m, add it to an average
                    % or anything else you want, then you could assign to 
                    % an output cell or matrix
                    % outputdata{m} = ...
                end
            end
        end     
    end
end