[Java] PrintWriter stops early

[eXorikos]

I'm writing a script that changes the format of a datafile into a more suitable format for us to import it in Origin. The problem is that it only writes until line 2179, but it is supposed to write until it reaches 4096, where the datafile also ends.

public static void main(String[] args) {
	try
	{
		BufferedReader inputStream = new BufferedReader(new FileReader("am0001.dat"));
		PrintWriter outputStream = new PrintWriter(new FileOutputStream("am0001.csv"));
		
		int count = 0;
		String line = inputStream.readLine();
		boolean quit = false;
		while(quit != true){
			if(line.equals("[ADC 1 Spectrum]")){
				outputStream.println("Channel;ADC 1");
				line = inputStream.readLine();
				int adc1 = 4096;
				while(count < adc1){
					count++;
					outputStream.println(count + ";" + line);
					line = inputStream.readLine();
				}
				quit = true;
				System.out.println("quit");
			}
			else{
				System.out.println(line);
				line = inputStream.readLine();
			}
		}
		System.out.println("The data was succesfully transferred to test.csv");
	}
	catch(FileNotFoundException e)
	{
		System.out.println("File opening problem.");
	}
	catch(IOException e)
	{
		System.out.println("Error reading from file test.dat");
	}
}

 

[Mark44]

I don't see anything jumping out at me in your code. Can you set a breakpoint for when count == 2178, and single step through to see why your code is choking when count is 2179? You're sure that the input file has 4096 lines?

 

[eXorikos]

It's a file from a Data System and the MCA goes to 4096 channels, so I can't see why it would dump the channels. Also I've added lines and these don't appear.

If I change the while to "while(count < 2179)" it stops at 1155...

I'm not that experienced and entering a breakpoint at even 1000 gives the same output...

 

[AlephZero]

It may or may not be interesting that the ratios 1155/2179 and 2179/4096 are approximately the same value, 0.53.

My guess is that Java and you have a different ideas about where the end-of-lines are in the input file.

I see you are adding line numbers to the output file. Are those line numbers where you expected them to be in the output file?

 

[Mark44]

It's a file from a Data System and the MCA goes to 4096 channels, so I can't see why it would dump the channels.

These terms don't mean anything to me. (I.e., I don't understand what you wrote.)

It might be that println is causing problems for you. I'm not sure what happens in this line of your code:
outputStream.println(count + ";" + line);
According to the documentation here, the println method can print an int or a string or several other types, but count, ";", and line are not all the same type. Possibly the compiler is treating the expression count + ";" + line as an Object. I don't know. It would be better to print count, and then println ";" + line.

Anyway, is the last line of output the same as the last line in your input file? If so, it might be the case that you are actually getting all the data in the input file, but println is not handling some of the line termination characters correctly. The documentation says

Prints formatted representations of objects to a text-output stream. This class implements all of the print methods found in PrintStream. It does not contain methods for writing raw bytes, for which a program should use unencoded byte streams.

Unlike the PrintStream class, if automatic flushing is enabled it will be done only when one of the println, printf, or format methods is invoked, rather than whenever a newline character happens to be output. These methods use the platform's own notion of line separator rather than the newline character.

Also I've added lines and these don't appear.

If I change the while to "while(count < 2179)" it stops at 1155...

I'm not that experienced and entering a breakpoint at even 1000 gives the same output...

 

[Borg]

Two questions come to mind.

1. Are you using the same input file for each test?
2. Is the line with "[ADC 1 Spectrum]" always the first line?

If #2 isn't true, it will not put any of the lines before that into the file and your loop will end early. Line reads in the else statement get printed to the output display but that's it. So, if [ADC 1 Spectrum] was around line 1917, the loop in the top section would only hit 2179 before the end of the file. That's the reason for the first question. If [ADC 1 Spectrum] shows up in different places in different files, you will get different numbers where it breaks.

 

[AlephZero]

I'm not sure what happens in this line of your code:
outputStream.println(count + ";" + line);

The + operator is the only Java operator that is overloaded, to mean concatenation for strings.

When one operand is a string object, the other operand is converted to a string and the two strings are concatenated.

So the OP's println statement won't cause any surprises.

 

[Borg]

Try this and tell us what the final print statement says. I rearranged the code with the assumption that there may be lines before the [ADC 1 Spectrum] line but, it works either way:

public static void main(String[] args) {
	try {
			BufferedReader inputStream = new BufferedReader(new FileReader("am0001.dat"));
			PrintWriter outputStream = new PrintWriter(new FileOutputStream("am0001.csv"));

			String line = inputStream.readLine();

			int preCount = 0;
			while(line != null && !line.equals("[ADC 1 Spectrum]")){
				System.out.println(preCount++ + line);
				line = inputStream.readLine();
			}

			int count = 0;
			outputStream.println("Channel;ADC 1");
			while(line != null){
				line = inputStream.readLine();
				outputStream.println(++count + ";" + line);
				line = inputStream.readLine();
			}

			System.out.println("There were " + count + " lines printed to the file and " + preCount +
				" lines before [ADC 1 Spectrum]");
		}
		catch (FileNotFoundException e) {
			System.out.println("File opening problem.");
		}
		catch (IOException e) {
			System.out.println("Error reading from file test.dat");
		}
}

 

[eXorikos]

I've discovered that everything works fine when I add flush() and close() statements in the end of the while-loop. Those were in the original code, but I had some more if tests inside, which I deleted. I must have deleted those statements by accident.

Thanks for the help!

 

[eXorikos]

The original file is formatted in the following way:

Info
[ADC 1]
0
123
8544
.
.
.
5616
0
0
[ADC 2]
56
65
312
.
.
.
564
6
3
[ADC 3]
.
.
.

and the point of the script is to get al the channels in comma separated format like this:

Info
Channel; ADC 1; ADC 2; ADC 3
1; 15; 123
2; 456; 456
3; 456; 96
...

My script now works for only ADC 1. How can I now add the other ADC to the lines?

 

[Mark44]

Give us a better idea of how the input file is formatted and what the output file should look like. For example, I don't see the connection between this section of the input file
[ADC 1]
0
123
8544
.
.
.
5616
0
0

and this output

Info
Channel; ADC 1; ADC 2; ADC 3
1; 15; 123

 

[eXorikos]

The inputfile is just formatted like it says there. The data system receives for every one of the 4096 channels a number. These are just sequentially put in the datafile. So in a line like it's stated in the. The first number for every ADC is the number for the first channel, the second fr the second channel. I'll match the number so it's more clear. I was typing it random. :)

Info
[ADC 1]
0
123
8544
.
.
.
5616
0
0
[ADC 2]
56
65
312
.
.
.
564
6
3
[ADC 3]
.
.
.

and the point of the script is to get al the channels in comma separated format like this:

Info
Channel; ADC 1; ADC 2; ADC 3
1; 0; 56; 
2; 123; 65; 
3; 8544; 312; 
...

I hope now it is more clear.

 

[Mark44]

If you're writing a comma-separated value (CSV) file, you should put commas (,)between the values. You are writing semicolons (;) which may or may not make a difference when someone tries to read the file.

It's a little more clear in that I see now how the numbers are selected from the data file, but there's still some that I don't get.

Is it a requirement that you output the data as you have shown? I.e., with the channel values going down a column?
If it's not a requirement, it would be much easier to format the data with the ADC numbers running down the first column, and channel values running across. IOW, like this:

ADC 1, 0, 123, 8544, ...
ADC 2, 56, 65, 312, ...
ADC 3, ...

 

[Grep]

I think you're going to need more logic in there to accumulate the values for each channel. Perhaps an ArrayList of strings, with one string per channel. You can get the string for a channel and append new values. And you can keep track of how many ADCs you've seen. I assume there's no holes in the ADC numbering (as in you won't go from ADC 1 to ADC 3)?

Once you've collected all the data, you can output everything by going through the ArrayList.

While we're on proper use of streams, getLine() returns a null on EOF. It should probably be checked for.

As a side note, not really comma separated if it uses semi-colons as a separator, no? :)

 

[eXorikos]

It's for importing in Origin and there you can choose semicolon. I'm just trying to get the same format as the datafile we have from a different datasystem with a different labview program.

I know this format works fine in Origin. I have no idea about different programs though, but that's not an issue.

 

[Grep]

This not being a homework/assignment question, perhaps the best way to show you how it can be done (and why your method won't work) is to show you code that works (or at least seems to). Either way, the different logic should be useful.

Keep in mind this was quickly produced and slightly hackish, and I would normally try and clean the logic up a little, and I would make it more fault tolerant. Still, it seems to do what you want.

Keep in mind I changed the name of the input file, so you'll need to change it back.

import java.io.*;
import java.util.*;

public class ADC {
   public static void main(String args[]) {
      ArrayList<String> data = new ArrayList<String>();
      try {
         BufferedReader inputStream = new BufferedReader(new FileReader("ADC.dat
"));
         PrintWriter outputStream = new PrintWriter(new FileOutputStream("am0001
.csv"));

         String line;
         int currentADC = 0;
         int currentChannelVal = 0;
         while ((line = inputStream.readLine()) != null) {
            // ADC label?
            if (line.length() > 0) {
               if (line.charAt(0) == '[') {
                  // Assuming there's no gaps in ADC numbering and starts at 1
                  // Done quickly, but should be done better than this
                  currentADC++;
                  currentChannelVal = 0;
               } else {
                  // Otherwise, it's a channel value
                  currentChannelVal++;
                  String channelStr;
                  if ((currentChannelVal-1) >= data.size()) {
                     data.add(line + "; ");
                  } else {
                     channelStr = data.get(currentChannelVal-1);
                     if (channelStr == null) {
                        data.set(currentChannelVal-1, line + "; ");
                     } else {
                        data.set(currentChannelVal-1, channelStr + line + "; ");
                     }
                  }
               }
            }
         }

         // Build output file
         for (int i = 0; i < currentChannelVal; i++) {
             outputStream.println((i+1) + "; " + data.get(i));
         }
         outputStream.close();
      } catch (Exception e) {
         e.printStackTrace();
      }
   }
}

 

[eXorikos]

In my first try I considered an ArrayList approach, but I thought this looked much cleaner. It does for only one ADC, but later I realized there would be more.

Maybe this way is also good. Make an ArrayList for every ADC and afterwards put them in output channel per channel. Hoping this wouldn't use an insane amount of memory or CPU for let's say 4 ADCs.