Skip to content

[zip] 7z complains about "Headers Error" when large files are added to a zip archive #623

Closed
@riton

Description

@riton

Hi,

Context

I'm trying to generate a zip archive of huge files (something like 2GB each).

The generated zip archive can successfully be extracted using my Ubuntu unzip command.
But it raises an error when I try to extract using my graphical user interface or 7z.

Here is the output of 7z in test integrity mode:

$ 7z t archive.zip

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,8 CPUs Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz (806EC),ASM,AES-NI)

Scanning the drive for archives:
1 file, 4294967634 bytes (4097 MiB)

Testing archive: archive.zip

ERRORS:
Headers Error

--
Path = archive.zip
Type = zip
ERRORS:
Headers Error
Physical Size = 4294967634
64-bit = +

              

Archives with Errors: 1

Open Errors: 1

Since unzip is able to extract the files, I would say that the archive is valid.
But I definitely does not understand why 7z is complaining here.

I've absolutely no experience in ZIP / ZIP64 archives and I may be using your library in a wrong way...

How to reproduce

I've used the following code to :

package main

import (
	"fmt"
	"io"
	"log"
	"os"
	"path/filepath"

	"github.com/klauspost/compress/zip"
)

var (
	// generated using "dd if=/dev/zero of=./file1 bs=1G count=2" or equivalent
	fileNames = []string{
		"./file1",
		"./file2",
	}
)

func main() {
	outFileW, err := os.Create("./archive.zip")
	if err != nil {
		log.Fatal(err)
	}
	defer outFileW.Close()

	zipWriter := zip.NewWriter(outFileW)

	for _, filename := range fileNames {

		fileToZip, err := os.Open(filename)
		if err != nil {
			log.Fatal(err)
		}
		defer fileToZip.Close()

		// Get the file information
		info, err := fileToZip.Stat()
		if err != nil {
			log.Fatal(err)
		}

		header, err := zip.FileInfoHeader(info)
		if err != nil {
			log.Fatal(err)
		}

		// Using FileInfoHeader() above only uses the basename of the file. If we want
		// to preserve the folder structure we can overwrite this with the full path.
		header.Name = filepath.Base(filename)

		// Change to store to avoid compression
		// see http://golang.org/pkg/archive/zip/#pkg-constants
		header.Method = zip.Store

		writer, err := zipWriter.CreateHeader(header)
		if err != nil {
			log.Fatal(err)
		}
		_, err = io.Copy(writer, fileToZip)
		if err != nil {
			log.Fatal(err)
		}
	}

	if err := zipWriter.Close(); err != nil {
		log.Fatal(err)
	}

	fmt.Println("archive.zip file created")
}

using

  • go 1.18.3
  • github.com/klauspost/compress v1.15.6

Test files are filled with zero and created using a command such as dd if=/dev/zero of=./file1 bs=1G count=2

Thanks in advance for your time and consideration

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions