How to store large rank-4 array to file?

In summary, the person is seeking advice on how to efficiently write operations within a larger code structure that involves calculating and manipulating a large set of matrix elements. They are experiencing issues with their current code, but have not provided enough information for others to help troubleshoot. The conversation also includes a rant about common bad coding practices in Fortran.
  • #1
gluons
15
0
I am writing a code to calculate and to operations with a large set of matrix elements, indexed by four numbers.

I am also writing these subroutines within a much larger code structure, so I do not have total freedom to modify the objects I am using.

I am trying something like this, which compiles but crashes. I am not sure if it would do what I want though, even if it didn't crash.

What is the best way to write operations like this? How can I most efficiently store the arrays and recall them?


inquire(iolength=recl) overlap
open(10,file='OVERLAPS.OUT',action='WRITE',form='UNFORMATTED',access='DIRECT',recl=recl)
do ikloc=1,nkptnrloc
ik=mpi_grid_map(nkpt,dim_k,loc=ikloc)
write(10,rec=ik) overlap(:,:,:,ikloc)
enddo
close(10)

inquire(iolength=recl) overlap
open(10,file='OVERLAPS.OUT',action='WRITE',form='UNFORMATTED',access='DIRECT',recl=recl)
do ikloc=1,nkptnrloc
ik=mpi_grid_map(nkpt,dim_k,loc=ikloc)
write(10,rec=ik) overlap(:,:,:,ikloc)
enddo
close(10)
 
Technology news on Phys.org
  • #2
gluons said:
I am writing a code to calculate and to operations with a large set of matrix elements, indexed by four numbers.

I am also writing these subroutines within a much larger code structure, so I do not have total freedom to modify the objects I am using.

I am trying something like this, which compiles but crashes. I am not sure if it would do what I want though, even if it didn't crash.

What is the best way to write operations like this? How can I most efficiently store the arrays and recall them?
Deleted extra copy of code.
gluons said:
inquire(iolength=recl) overlap
open(10,file='OVERLAPS.OUT',action='WRITE',form='UNFORMATTED',access='DIRECT',recl=recl)
do ikloc=1,nkptnrloc
ik=mpi_grid_map(nkpt,dim_k,loc=ikloc)
write(10,rec=ik) overlap(:,:,:,ikloc)
enddo
close(10)

Others on this site are more versatile with fortran, as the last time I wrote any code of this type was about 20 years ago.

In any case, I don't believe you have given us enough information to be able to help. The problem you describe may or may not be occurring in the lines of code you show, and might be a result of something in the code that you don't show.

From your description your program is syntactically correct (it compiles successfully) but a logical error is causing problems at run time. Simply saying that the program "crashed" gives us no clue at all about why the crash occurred. Any additional information that the program produces at that time would be helpful for us to determine the cause.

If you have a debugger to use, that could be used to narrow down the cause of the crash. If you are unfamiliar with using debuggers, the old-fashioned way to find problems is to liberally sprinkle WRITE statements throughout the code to display the values of variables when the code is running.

<rant>
I mean no disrespect, but your coding style is pretty much identical to the coding style used by countless Fortran programmers. Here are some of the hallmarks of bad Fortran code.

  1. Incomprehensible variable names - ikloc, nkptnrloc, ik.
    I have no idea what these are supposed to represent, which makes it impossible to understand what their roles might be or the values they might have. I can guess that ikloc represents the location of something - ik? - but that's only a guess. Possibly nkptnrloc represents the location of something, but I can't even guess what an nkptnr might be.
    At one point many years ago, Fortran programmers were constrained to using short variable names due to the lack of memory. Although those days are long gone, all too many Fortran programmers continue following this style.
  2. Nonuse of whitespace - The statement below has not a single space character in it.
    Code:
    open(10,file='OVERLAPS.OUT',action='WRITE',form='UNFORMATTED',access='DIRECT',recl=recl)
    In general, it seems to me that Fortran programmers are unfamiliar with the location and purpose of the space bar. Whitespace added to program code makes it easier for us humans to read and understand the code. The lack of whitespace and the gobbledegook variable names puts the code pretty firmly into the "write-only" category, IMO.
Other traits that I often see in others' Fortran code are no indentation or random indentation of program control structures, and lack of comments.

Although the compiler couldn't care less about the names of variables, use of whitespace, indentation, and comments, we poor humans have a more difficult time understanding what the code is trying to do when little or no thought is given to these attributes of good programming style.
</rant>
 

Related to How to store large rank-4 array to file?

1. How do I store a large rank-4 array to file?

To store a large rank-4 array to file, you can use the numpy.save function to save the array as a binary file. This function takes in the array as the first argument and the file name as the second argument. Once saved, you can use the numpy.load function to load the array from the file.

2. What is the best file format for storing a large rank-4 array?

The best file format for storing a large rank-4 array depends on your specific needs. If you need a file format that is efficient in terms of storage space, a binary file format like .npy or .npz would be a good choice. If you need a file format that can be easily read by other programs, a text-based format like .csv or .txt may be a better option.

3. Can I compress a large rank-4 array before storing it to file?

Yes, you can compress a large rank-4 array before storing it to file. You can use the numpy.savez_compressed function to save the compressed array as a .npz file. This function takes in the array as the first argument and the file name as the second argument. When loading the array from the file, use the numpy.load function and specify allow_pickle=True to load the compressed array.

4. How can I ensure the integrity of my stored large rank-4 array?

To ensure the integrity of your stored large rank-4 array, you can use the numpy.save or numpy.savez functions with the allow_pickle=True argument. This will save a checksum of the array along with the data, allowing you to check for any data corruption when loading the array from the file. You can also use file compression as an additional means of protecting the integrity of your data.

5. Are there any limitations to storing a large rank-4 array to file?

There are a few limitations to consider when storing a large rank-4 array to file. One limitation is the available storage space on your device. If the array is too large, you may not have enough space to store it. Another limitation is the capabilities of your computer's hardware, such as memory and processing power. If the array is too large, it may exceed your computer's capabilities and cause errors or slow performance. It's important to consider these limitations before attempting to store a large rank-4 array to file.

Similar threads

  • Programming and Computer Science
Replies
6
Views
1K
  • Programming and Computer Science
Replies
13
Views
2K
  • Programming and Computer Science
Replies
2
Views
2K
  • Engineering and Comp Sci Homework Help
Replies
7
Views
1K
  • Programming and Computer Science
Replies
10
Views
25K
  • Programming and Computer Science
Replies
1
Views
5K
  • Programming and Computer Science
Replies
4
Views
3K
  • Programming and Computer Science
Replies
4
Views
11K
  • Programming and Computer Science
Replies
5
Views
3K
  • Programming and Computer Science
Replies
2
Views
2K
Back
Top