Bug 405918 - Craft INSTALL instructions don't work
Summary: Craft INSTALL instructions don't work
Status: RESOLVED FIXED
Alias: None
Product: kdiff3
Classification: Applications
Component: application (show other bugs)
Version: 1.8.x
Platform: Microsoft Windows Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: michael
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-03-27 03:48 UTC by Ross Boylan
Modified: 2019-05-24 21:01 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In: 1.8.2


Attachments
attachment-16322-0.html (671 bytes, text/html)
2019-04-12 19:17 UTC, michael
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ross Boylan 2019-03-27 03:48:55 UTC
SUMMARY
INSTALL instructions don't work.  Build fails to find the first package and gives up.


STEPS TO REPRODUCE
1. Installed craft on Win 10 with MSVC 2017.
2. Followed new INSTALL instructions from https://cgit.kde.org/kdiff3.git/tree/INSTALL (blob: 88471cbe3d4c11007abf253ee9c60c54ad0a2287).
Since I'd already followed most of the steps (except I did not use Admin privileges, and CraftRoot was in my local directory hierarchy). This came down to executing ...
3. Craft --package kdiff3

OBSERVED RESULT
PS C:\Users\rdboylan\Documents\src\CraftRoot> Craft --package kdiff3
Craft               : C:\Users\rdboylan\Documents\src\CraftRoot
Version             : master
ABI                 : windows-msvc2017_64-cl
Download directory  : C:\Users\rdboylan\Documents\src\CraftRoot\download
===========================================================================
*** Handling package: extragear/kdiff3, action: package ***
*** Action: package for extragear/kdiff3 ***
could not find installed nsis package, 'craft nsis'
Action: package for extragear/kdiff3:master FAILED
*** Craft package failed: extragear/kdiff3 after 1 second ***
PS C:\Users\rdboylan\Documents\src\CraftRoot>

EXPECTED RESULT
Download and build all required packages.
I thought craft would always fetch packages it didn't find; I'm not sure why it just gave up.


SOFTWARE/OS VERSIONS
Windows: 10

ADDITIONAL INFORMATION
MSVS 2019 Preview is also on the system.
The craft setup appeared to have been successful, but this is the first time I've used it for anything other than building itself.
craft has some local modifications to support VS2019; I don't think they are material for this scenario.

Thank you for providing the INSTALL document in response to my earlier request.
Comment 1 michael 2019-03-29 20:44:50 UTC
Try the fallowing:
1)cd to your craft root
Run:
2)C:\CraftRoot\craft\craftenv.ps1
3)craft kdiff3
4)craft --package kdiff3

The craft root I was running still had build products from the last run.
Comment 2 michael 2019-03-29 22:26:17 UTC
On closer inspection you have something more going on than just the missing command. I am going to cc our windows support list. MSVC 2019 support is being worked on part of https://bugs.kde.org/show_bug.cgi?id=405662.

Could someone familiar with craft internals have a look at this please?
Comment 3 Ross Boylan 2019-03-29 23:24:30 UTC
craft kdiff3 did a lot more, but had a hiccup along the way and then a heart attack:

Hiccup:
*** Craft all succeeded: libs/dbus after 5 seconds ***
===============================================================================================================================================================================================
*** Handling package: binary/mysql, action: all ***
*** Action: fetch-binary for binary/mysql ***
wget https://files.kde.org/craft/master/Qt_5.12.2/windows/msvc2017_64/cl/RelWithDebInfo/binary/mysql/mysql-5.7.18-7-20190319T110634-windows-msvc2017_64-cl.7z
C:/Users/rdboylan/Documents/src/CraftRoot/downl 100%[======================================================================================================>]  60.38M  17.5MB/s    in 3.3s
executing command: C:\Users\rdboylan\Documents\src\CraftRoot\dev-utils\bin\7za.exe x -r -y -oC:\Users\rdboylan\Documents\src\CraftRoot\build\binary\mysql\image-RelWithDebInfo-5.7.18 C:\Users\rdboylan\Documents\src\CraftRoot\download\cache\Qt_5.12.2\windows\msvc2017_64\cl\RelWithDebInfo\binary\mysql\mysql-5.7.18-7-20190319T110634-windows-msvc2017_64-cl.7z -t7z -bso2 -bsp1
executing command: C:\Users\rdboylan\Documents\src\CraftRoot\bin\mysqld.exe --console --initialize-insecure
mysqld: Could not create or access the registry key needed for the MySQL application
to log to the Windows EventLog. Run the application with sufficient
privileges once to create the key, add the key manually, or turn off
logging for that application.
2019-03-29T21:44:15.689166Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2019-03-29T21:44:15.690169Z 0 [ERROR] Cannot open Windows EventLog; check privileges, or start server with --log_syslog=0
2019-03-29T21:44:18.180819Z 0 [Warning] InnoDB: New log files created, LSN=45790
2019-03-29T21:44:18.823955Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2019-03-29T21:44:19.129020Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: ce849c47-526b-11e9-aca6-b88584b6a8d5.
2019-03-29T21:44:19.225276Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2019-03-29T21:44:19.233296Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
*** Craft all succeeded: binary/mysql after 1 minute 18 seconds ***
===============================================================================================================================================================================================
Comments:
  1) Running without elevated privileges seems the likely immediate cause;
  2) But it seems odd that elevated privileges should be required for what I understand to be a build form source step;
  3) It's unclear to me what the status is at the end.  On the one hand, there is [ERROR} in the middle of the log.  On the other hand the final message indicates success.

At any rate, execution continued for quite awhile.  But then
*** Craft all succeeded: kde/frameworks/tier1/kcodecs after 15 seconds ***
===============================================================================================================================================================================================
*** Handling package: libs/boost/boost-headers, action: all ***
*** Action: fetch-binary for libs/boost/boost-headers ***
*** libs/boost/boost-headers not found in cache ***
*** Action: fetch for libs/boost/boost-headers ***
wget https://dl.bintray.com/boostorg/release/1.69.0/source/boost_1_69_0.tar.gz
C:/Users/rdboylan/Documents/src/CraftRoot/downl 100%[======================================================================================================>] 106.53M  41.6MB/s    in 2.6s
*** Action: unpack for libs/boost/boost-headers ***
Failed to unpack boost_1_69_0.tar.gz
Traceback (most recent call last):
  File "C:\Users\rdboylan\Documents\src\CraftRoot\craft\bin\utils.py", line 86, in unpackFile
    shutil.unpack_archive(os.path.join(downloaddir, filename), workdir)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\shutil.py", line 999, in unpack_archive
    func(filename, extract_dir, **kwargs)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\shutil.py", line 934, in _unpack_tarfile
    tarobj.extractall(extract_dir)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\tarfile.py", line 2002, in extractall
    numeric_owner=numeric_owner)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\tarfile.py", line 2044, in extract
    numeric_owner=numeric_owner)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\tarfile.py", line 2114, in _extract_member
    self.makefile(tarinfo, targetpath)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\tarfile.py", line 2155, in makefile
    with bltn_open(targetpath, "wb") as target:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\rdboylan\\Documents\\src\\CraftRoot\\build\\_\\20653f50\\boost_1_69_0\\libs\\geometry\\doc\\html\\geometry\\reference\\spatial_indexes\\boost__geometry__index__rtree\\rtree_parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const___.html'
Action: unpack for libs/boost/boost-headers:1.69.0 FAILED
*** Craft all failed: libs/boost/boost-headers after 5 minutes 30 seconds ***
fatal error: package libs/boost/boost-headers all failed
PS C:\Users\rdboylan\Documents\src\CraftRoot>

It sat for awhile, I think at the wget, before failing.  Checking now, the file referenced in "No such file" is not there, though the directory it is in does exist.  The only file in it is rtree__.html.  It looks as if \rtree_parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const___.html might be an error, with the argument parameters turned into text.
C:\Users\rdboylan\Documents\src\CraftRoot\download\archives\libs\boost\boost-headers\boost_1_69_0.tar.gz exists with c 109MB.
gzip --test says the file is OK.  And the URL used for downloading seems to point at a valid location.  There is a boatload of security software on the system, mostly not under my control.  It definitely slows things down and it might have blocked access to some  file.

BTW, I'm the one working on the VS2019 code for craft.  This definitely does not mean I'm knowledgeable about craft.

Ross
________________________________________
From: michael <bugzilla_noreply@kde.org>
Sent: Friday, March 29, 2019 1:44 PM
To: Boylan, Ross
Subject: [kdiff3] [Bug 405918] INSTALL instructions don't work

https://bugs.kde.org/show_bug.cgi?id=405918

--- Comment #1 from michael <reeves.87@gmail.com> ---
Try the fallowing:
1)cd to your craft root
Run:
2)C:\CraftRoot\craft\craftenv.ps1
3)craft kdiff3
4)craft --package kdiff3

The craft root I was running still had build products from the last run.

--
You are receiving this mail because:
You reported the bug.
Comment 4 Hannah von Reth 2019-03-29 23:34:45 UTC
For the boost tar.
Please enable Windows dev mode https://docs.microsoft.com/en-us/windows/uwp/get-started/enable-your-device-for-development so that symlinks are supported(then we can use 7z to unpack stuff)
Comment 5 Ross Boylan 2019-03-30 00:19:04 UTC
I enabled developer mode and reran.
First time failed with
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\rdboylan\\Documents\\src\\CraftRoot\\build\\_\\20653f50\\boost_1_69_0\\libs'

I made sure I had nothing open on the directory tree and tried again.  I still got
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\rdboylan\\Documents\\src\\CraftRoot\\build\\_\\20653f50\\boost_1_69_0\\libs\\geometry\\doc\\html\\geometry\\reference\\spatial_indexes\\boost__geometry__index__rtree\\rtree_parameters_type_const____indexable_getter_const____value_equal_const____allocator_type_const___.html'

Do I need to delete the directory (which?)?  Restart powershell after having set dev mode (don't think so: tried it and got the 2nd error above)? Do some equivalent of make clean, hopefully one that will only affect boost-headers?

Except for the problem with qith MySQL, the stuff before this all seems to have built OK.  Does that fact I was not in dev mode means some of those packages might not be quite right either?

Ross
________________________________________
From: Hannah von Reth <bugzilla_noreply@kde.org>
Sent: Friday, March 29, 2019 4:34 PM
To: Boylan, Ross
Subject: [kdiff3] [Bug 405918] INSTALL instructions don't work

https://bugs.kde.org/show_bug.cgi?id=405918

Hannah von Reth <vonreth@kde.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |vonreth@kde.org

--- Comment #4 from Hannah von Reth <vonreth@kde.org> ---
For the boost tar.
Please enable Windows dev mode
https://protect2.fireeye.com/url?k=9f8ede37-c3ceb361-9f8ef92a-0cc47adb57f0-5e9fc5eb173cc7c9&u=https://docs.microsoft.com/en-us/windows/uwp/get-started/enable-your-device-for-development
so that symlinks are supported(then we can use 7z to unpack stuff)

--
You are receiving this mail because:
You reported the bug.
Comment 6 Hannah von Reth 2019-03-30 09:06:41 UTC
Can you try to manually delete the boost folder?
Looks like some bad files, created from the failed fallback unpack.
Comment 7 Ross Boylan 2019-03-30 18:13:43 UTC
I deleted both the boost folder and the one above it, 20653f50.  This did allow the build to continue.  After > 30 minutes it finished, showing success.  The next step fails, however, in the same way it did before.

craft --package kdiff3:
===============================================================================================================================================================================================
*** Handling package: extragear/kdiff3, action: package ***
*** Action: package for extragear/kdiff3 ***
could not find installed nsis package, 'craft nsis'
Action: package for extragear/kdiff3:master FAILED
*** Craft package failed: extragear/kdiff3 after 0 seconds ***


cs kdiff3 does seem to put me in the source directory, which looks like a git repo.
cb kdiff3 puts me in a subdirectory, and from there bin\kdiff3 launches the program! Yea!
Ross
________________________________________
From: Hannah von Reth <bugzilla_noreply@kde.org>
Sent: Saturday, March 30, 2019 2:06:41 AM
To: Boylan, Ross
Subject: [kdiff3] [Bug 405918] INSTALL instructions don't work

https://bugs.kde.org/show_bug.cgi?id=405918

--- Comment #6 from Hannah von Reth <vonreth@kde.org> ---
Can you try to manually delete the boost folder?
Looks like some bad files, created from the failed fallback unpack.

--
You are receiving this mail because:
You reported the bug.
Comment 8 Hannah von Reth 2019-03-31 11:31:05 UTC
I'd suggest that you read the message you get for --package
Comment 9 Ross Boylan 2019-03-31 17:58:05 UTC
Well, the message just says a package isn't installed.  The immediate point is that the install instructions for kdiff3 are still incomplete.

I don't know if it's a kdiff3 or, more likely, a craft issue, but it seems odd that craft doesn't go ahead and install the package nsis.

If the intent of the message from craft was that I should execute 'craft nsis', that is not clear.  That phrase currently reads as an appositive for "nsis package" NOT as an instruction.  Admittedly it doesn't quite make sense in that reading.

On the theory that it was intended as an instruction (which for some reason craft --package didn't want to execute) I executed 'craft nsis' and then repeated the craft --package command.  I think this worked.

So does the line
Output: "C:\Users\rdboylan\Documents\src\CraftRoot\tmp\kdiff3-prefilter-b794c83-windows-msvc2017_64-cl.exe"
mean that file is an installer for the kdiff3 I  previously built?

Here's the transcript of the final, --package, step:
==============================================================================================================================================================================================
*** Handling package: extragear/kdiff3, action: package ***
*** Action: package for extragear/kdiff3 ***
Overriding C:\Users\rdboylan\Documents\src\CraftRoot\build\extragear\kdiff3\archive\bin\data\info\dir
Overriding C:\Users\rdboylan\Documents\src\CraftRoot\build\extragear\kdiff3\archive\bin\data\info\dir
executing command: C:\Users\rdboylan\Documents\src\CraftRoot\dev-utils\bin\7za.exe a -r C:\Users\rdboylan\Documents\src\CraftRoot\tmp\kdiff3-prefilter-b794c83-windows-msvc2017_64-cl.7z -bso2 -bsp1 C:\Users\rdboylan\Documents\src\CraftRoot\build\extragear\kdiff3\archive\*
executing command: C:\Users\rdboylan\Documents\src\CraftRoot\dev-utils\bin\7za.exe a -r C:\Users\rdboylan\Documents\src\CraftRoot\tmp\kdiff3-prefilter-b794c83-windows-msvc2017_64-cl-dbg.7z -bso2 -bsp1 C:\Users\rdboylan\Documents\src\CraftRoot\build\extragear\kdiff3\archive-dbg\*


executing command: C:\Users\rdboylan\Documents\src\CraftRoot\dev-utils\bin\makensis.exe /V3 C:\Users\rdboylan\Documents\src\CraftRoot\build\_\51b6ca1c\kdiff3.nsi
MakeNSIS v3.03 - Copyright 1999-2018 Contributors
See the file COPYING for license details.
Credits can be found in the Users Manual.

Processing config: C:\Users\rdboylan\Documents\src\CraftRoot\dev-utils\nsis\nsisconf.nsh
Processing script file: "C:\Users\rdboylan\Documents\src\CraftRoot\build\_\51b6ca1c\kdiff3.nsi" (ACP)

Processed 1 file, writing output (x86-ansi):

Output: "C:\Users\rdboylan\Documents\src\CraftRoot\tmp\kdiff3-prefilter-b794c83-windows-msvc2017_64-cl.exe"
Install: 6 pages (384 bytes), 2 sections (1 required) (4144 bytes), 822 instructions (23016 bytes), 295 strings (5369 bytes), 1 language table (334 bytes).
Uninstall: 2 pages (192 bytes), 1 section (2072 bytes), 328 instructions (9184 bytes), 163 strings (2735 bytes), 1 language table (254 bytes).
Datablock optimizer saved 1888 bytes (~0.0%).

Using zlib compression.

EXE header size:               63488 / 37888 bytes
Install code:                   5929 / 31663 bytes
Install data:               74342779 / 75050374 bytes
Uninstall code+data:           24201 / 28522 bytes
CRC (0x3C024D03):                  4 / 4 bytes

Total size:                 74436401 / 75148451 bytes (99.0%)
*** Craft package succeeded: extragear/kdiff3 after 5 minutes 14 seconds ***
Comment 10 michael 2019-04-12 19:17:36 UTC
Created attachment 119388 [details]
attachment-16322-0.html

C:\Users\rdboylan\Documents\src\CraftRoot\tmp\kdiff3-prefil
ter-b794c83-windows-msvc2017_64-cl.exe

That would be the install file generated by this.
Comment 11 michael 2019-05-24 20:45:39 UTC
Git commit e77ed000c50ae53d46721b72c7e35c7781f45c38 by Michael Reeves.
Committed on 24/05/2019 at 20:29.
Pushed by mreeves into branch '1.8'.

Fix craft install instruction
FIXED-IN: 1.8.2

M  +8    -3    INSTALL

https://commits.kde.org/kdiff3/e77ed000c50ae53d46721b72c7e35c7781f45c38
Comment 12 michael 2019-05-24 21:01:06 UTC
Git commit 6557f35d70494ce4b02c3d09a6dd5b01d0207aea by Michael Reeves.
Committed on 24/05/2019 at 20:47.
Pushed by mreeves into branch 'master'.

Fix craft install instruction
FIXED-IN: 1.8.2

M  +8    -3    INSTALL

https://commits.kde.org/kdiff3/6557f35d70494ce4b02c3d09a6dd5b01d0207aea