Search code examples
pythonnumpyfor-loopimportaddition

Read in Values from external file, add them, print results


Problem: I have 50 text files, each with thousands of lines of text, each line has a value on it. I am only interesting in a small section near the middle (lines 757-827 - it is actually lines 745-805 I'm interested in, but the first 12 lines of every file is irrelevant stuff). I would like to read each file in. And then total the values between those lines. In the end I would like it to print off a pair of numbers in the format (((n+1)*18),total count), where n is the number of the file (since they are numbered starting at zero). Then repeat for all 50 files, giving 50 pairs of numbers, looking something like:

(18,77),(36,63),(54,50),(72,42),...

Code:

import numpy as np
%matplotlib inline
from numpy import loadtxt, linspace
import glob, os

fileToRun = 'Run0'
location = 'ControlRoom6'
DeadTime = 3
LiveTime = 15 

folderId = '\\'
baseFolder = 'C:'+folderId+'Users'+folderId+location+folderId+'Documents'+folderId+'PhD'+folderId+'Ubuntu-Analysis-DCF'+folderId+'DCF-an-b+decay'+folderId+'dcp-ap-27Al'+folderId+''
prefix = 'DECAY_COINC'

folderToAnalyze = baseFolder + fileToRun + '\\'
MaestroT = LiveTime + DeadTime

## Gets number of files
files = []
os.chdir(folderToAnalyze)
for file in glob.glob(prefix + "*.Spe"):
    files.append(file)
numfiles = len(files)
if numfiles<=1:
    print('numfiles is {0}, minimum of 2 is required'.format(numfiles))
    raise SystemExit(0)
    
    xmin = 745
    xmax = 815
    skips = 12
    
    n=[]
    count=[]
    
    for n in range(0, numfiles):
    
        x = np.linspace(0, 8191, 8192)
        finalprefix = str(n).zfill(3)
        fullprefix = folderToAnalyze + prefix + finalprefix
        y = loadtxt(fullprefix + ".Spe", skiprows = 12, max_rows = 8192) 
    
        
        for x in range(xmin+skips,xmax+skips):
            count = count + y
            time = MaestroT*(n+1)
            
            print(time, count)

Current output is: 'ValueError Traceback (most recent call last) in 84 85 for x in range(xmin+skips,xmax+skips): ---> 86 count = count + y 87 time = MaestroT*(n+1) 88

ValueError: operands could not be broadcast together with shapes (0,) (8192,)'

However I did previously have this running, it just printing out thousands of seemingly unconnected numbers. Does anyone know how I can alter the code to acheive the desired result?

EDIT: Data Set

In order to make the example easier to use, I've made a dropbox with some dummy data. The files are named the same as it would be reading in, and are written in the same format (the first 12 rows with unuseful information). Link is Here. I haven't written 8192 dummy numbers as I thought it would probably be easier and produce a nearer facsilime to just use the actual files with a few numbers changed.


Solution

  • Solution was to edit code as shown starting from 'xmin = 745':

    xmin = 745
    xmax = 815
    skip = 12
    
    for n in range(0, numfiles):
    
        total = 0
        
        x = np.linspace(0, 8191, 8192)
        finalprefix = str(n).zfill(3)
        fullprefix = folderToAnalyze + prefix + finalprefix
        y = loadtxt(fullprefix + ".Spe", skiprows= xmin + skip, max_rows = xmax - xmin)
    
        
        for x in y:
            val = int(x)
            total = total + val
        
        print(((n+1)*MaestroT), total)
    

    Prints out as

    18 74

    36 64

    54 62

    72 54

    90 47

    108 39

    126 40

    144 35

    etc. Which fit my needs.