Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable unpack/repack of RPM containers #15405

Merged
merged 8 commits into from
Jan 15, 2025
Merged

Enable unpack/repack of RPM containers #15405

merged 8 commits into from
Jan 15, 2025
+355 −5

Conversation

NikolaMilosavljevic
Copy link
Member

Fixes #14437

This work enables unpack and repack of .deb containers. I've tested this with all current .NET RPM packages. As the code requires the use of cpio tool, it is conditioned to only work on Linux.

@NikolaMilosavljevic
Copy link
Member Author

Looks like some bad auto-merge causing build issues. Will fix this soon.

@NikolaMilosavljevic @mmitche
Update src/Microsoft.DotNet.SignTool.Tests/SignToolTests.cs
11a9dcf 
Co-authored-by: Matt Mitchell <mmitche@microsoft.com>
mmitche
mmitche reviewed Jan 13, 2025
View reviewed changes
src/Microsoft.DotNet.SignTool/src/BatchSignUtil.cs
else if (!_signTool.VerifySignedRpm(log, file.FullPath))
{
_log.LogError($"Deb package {file.FullPath} is not signed properly.");
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add an else here with a "signed properly" message?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add it for both DEB and RPM.

@build-analysis build-analysis bot mentioned this pull request Jan 14, 2025
Open
3 tasks
ellahathaway
ellahathaway reviewed Jan 14, 2025
View reviewed changes
src/Microsoft.DotNet.SignTool.Tests/SignToolTests.cs
@@ -34,6 +35,7 @@ public class SignToolTests : IDisposable
{".psc1", new List<SignInfo>{ new SignInfo("PSCCertificate") } },
{".dylib", new List<SignInfo>{ new SignInfo("DylibCertificate") } },
{".deb", new List<SignInfo>{ new SignInfo("LinuxSign") } },
{".rpm", new List<SignInfo>{ new SignInfo("LinuxSign") } },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the RPM extension also need to be added to s_fileExtensionSignInfoWithCollisionId, s_fileExtensionSignInfoPostBuild, and SignableExtensions? It's possible that this may be addressed in #14433

src/Microsoft.DotNet.SignTool/src/SignToolConstants.cs Show resolved Hide resolved
src/Microsoft.DotNet.SignTool/src/ZipData.cs Show resolved Hide resolved
src/Microsoft.DotNet.SignTool/src/ZipData.cs Show resolved Hide resolved
@build-analysis build-analysis bot mentioned this pull request Jan 14, 2025
Open
2 tasks
@NikolaMilosavljevic
Copy link
Member Author

Hmm, Windows tests are failing as they're trying to run the RPM-specific test. Helix log indicates that this is executed on Linux and both commands are failing: https://helixr1107v0xdeko0k025g8.blob.core.windows.net/dotnet-arcade-refs-pull-15405-merge-c0a2b5fcd9fa4337af/Microsoft.DotNet.SignTool.Tests.dll/2/console.5f9debda.log?helixlogtype=result

Console log: 'Microsoft.DotNet.SignTool.Tests.dll' from job c0a2b5fc-d9fa-4337-af25-e28ec6a8a902 (ubuntu.2204.amd64.open) using docker image mcr.microsoft.com/dotnet-buildtools/prereqs:almalinux-8-helix-amd64 on a008SX5

bash: cpio: command not found
find: 'standard output': Broken pipe
find: write error
find: 'file': No such file or directory

Linux tests are failing with the same issue - https://helixr1107v0xdeko0k025g8.blob.core.windows.net/dotnet-arcade-refs-pull-15405-merge-52fab730ffb84b0d9b/Microsoft.DotNet.SignTool.Tests.dll/2/console.cb9f2381.log?helixlogtype=result

@NikolaMilosavljevic
Copy link
Member Author

NikolaMilosavljevic commented Jan 14, 2025 •
edited
Loading

Before I add cpio and file packages to helix almalinux image, I wonder if there is something wrong with test jobs in arcade. Each test leg (Windows and Linux) sends for testing to three targets, Windows, Ubuntu and almalinux. This seems wrong.

From: https://dev.azure.com/dnceng-public/public/_build/results?buildId=917110&view=logs&j=21143b0a-ace1-5058-215e-b11404388d17&t=b6047ca6-0230-5e90-2195-54379d550513

  Job c0a2b5fc-d9fa-4337-af25-e28ec6a8a902 on (AlmaLinux.8.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:almalinux-8-helix-amd64 is completed with 24 finished work items.
  Job f653c336-8b5d-4237-bebb-5c589e318064 on (Debian.12.Amd64.Open)ubuntu.2204.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-amd64 is completed with 24 finished work items.
  Job d39cdf1a-e378-4f8f-a6e0-af041e3208e5 on Windows.10.Amd64.Open is completed with 24 finished work items.

This PR is failing the RPM test on almalinux and debian 12 and skipped on Windows.

@mmitche

@NikolaMilosavljevic
Copy link
Member Author

I definitely need to add cpio and file packages to debian-12-helix-amd64 image.

@NikolaMilosavljevic
Copy link
Member Author

Before I add cpio and file packages to helix almalinux image, I wonder if there is something wrong with test jobs in arcade. Each test leg (Windows and Linux) sends for testing to three targets, Windows, Ubuntu and almalinux. This seems wrong.

From: https://dev.azure.com/dnceng-public/public/_build/results?buildId=917110&view=logs&j=21143b0a-ace1-5058-215e-b11404388d17&t=b6047ca6-0230-5e90-2195-54379d550513

  Job c0a2b5fc-d9fa-4337-af25-e28ec6a8a902 on (AlmaLinux.8.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:almalinux-8-helix-amd64 is completed with 24 finished work items.
  Job f653c336-8b5d-4237-bebb-5c589e318064 on (Debian.12.Amd64.Open)ubuntu.2204.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-amd64 is completed with 24 finished work items.
  Job d39cdf1a-e378-4f8f-a6e0-af041e3208e5 on Windows.10.Amd64.Open is completed with 24 finished work items.

This PR is failing the RPM test on almalinux and debian 12 and skipped on Windows.

@mmitche

All 3 Helix queues are run regardless of the job target (Windows vs Linux) due to:

arcade/tests/UnitTests.proj

Lines 61 to 65 in 2696b08

<ItemGroup Condition=" '$(HelixAccessToken)' != '' ">
<HelixTargetQueue Include="(AlmaLinux.8.Amd64)Ubuntu.2204.Amd64@mcr.microsoft.com/dotnet-buildtools/prereqs:almalinux-8-helix-amd64"/>
<HelixTargetQueue Include="Windows.10.Amd64"/>
<HelixTargetQueue Include="(Debian.12.Amd64)ubuntu.2204.amd64@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-amd64"/>
</ItemGroup>

@NikolaMilosavljevic NikolaMilosavljevic mentioned this pull request Jan 14, 2025
Merged
@NikolaMilosavljevic
Copy link
Member Author

/azp run

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@NikolaMilosavljevic
Copy link
Member Author

All checks are now passing - I've updated the Helix images to add the missing packages with dotnet/dotnet-buildtools-prereqs-docker#1322

@NikolaMilosavljevic NikolaMilosavljevic merged commit fcbe4fd into dotnet:main Jan 15, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ellahathaway ellahathaway ellahathaway approved these changes

@mmitche mmitche mmitche approved these changes

@jkoritzinsky jkoritzinsky Awaiting requested review from jkoritzinsky

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Enable unpack/repack of .rpm containers
3 participants
@NikolaMilosavljevic @mmitche @ellahathaway