Binary sources

[lscheffler]

@DougHennig

Hi Doug,

I like to remove the binary sources.

• it's just bloat
• as we see in other repos, it makes merging nearly impossible
• the code to recreate the binaries comes with the repo, since the main program is a prg.
• A program to regenerate (ReCreate_FoxBin2Prg.prg) is enclosed, see
  • readme for exe use (if one need this, from VFP the prg is fine), or development
  • contributing
• The Thor version comes with an own set of files, independent of a full download
• There are not so many developing this repo, and if, they should understand FoxBin2Prg?

It adds one additional step after anything like git switch, checkout, merge or the like - one must regenerate, or the VFP sources are wrong. But the advantage is so huge.

[DougHennig]

That seems reasonable.

[RickBorup]

Hi Lutz, There are at least two good reasons to include the binaries in the repo. 1. Anyone who has already cloned the repo and subsequently pulls from the remote to get the latest will quite reasonably assume they have the latest binaries. If the binaries are not in the remote repo, an extra step in required to regenerate them from the ??2 file, and this can easily be overlooked. 2. All it takes is one time when there is an error regenerating the binary to make you wish you had included them in the repo. When this happens you must either find a way to resolve the error, typically by some modification to the ??2 or even a fix to FoxBin2PRG, or you must rely on having a backup copy of the binary somewhere. While including the binaries contributes to bloat, the size of the repository is not an issue with a small project like FoxBin2PRG. I recommend leaving the binaries in the repo.

…

-Rick

________________________________ Rick Borup Information Technology Associates, LLC ***@***.***> ***@***.*** <http://www.ita-software.com/> www.ita-software.com From: Lutz Scheffler ***@***.***> Sent: Monday, December 4, 2023 6:33 AM To: fdbozzo/foxbin2prg ***@***.***> Cc: Subscribed ***@***.***> Subject: [fdbozzo/foxbin2prg] Binary sources (Discussion #103) @DougHennig <https://github.com/DougHennig> Hi Doug, I like to remove the binary sources. * it's just bloat * as we see in other repos, it makes merging nearly impossible * the code to recreate the binaries comes with the repo, since the main program is a prg. * A program to regenerate (ReCreate_FoxBin2Prg.prg) is enclosed, see * readme <https://github.com/lscheffler/foxbin2prg#readme> for exe use (if one need this, from VFP the prg is fine), or development * contributing <https://github.com/lscheffler/foxbin2prg/blob/fork_mod/.github/CONTRIBUTING.md> * The Thor version comes with an own set of files, independent of a full download * There are not so many developing this repo, and if, they should understand FoxBin2Prg? It adds one additional step after anything like git switch, checkout, merge or the like - one must regenerate, or the VFP sources are wrong. But the advantage is so huge. — Reply to this email directly, view it on GitHub <#103> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACMHRZLQGLJSOKLO4J57YPLYHW7GZAVCNFSM6AAAAABAF4YWDKVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZVHEZDENRQGY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- This email has been checked for viruses by Avast antivirus software. www.avast.com

[lscheffler]

Hi @RickBorup,

In general

The general rules why no binary source (including tables and databases)

• FoxBin2Prg runs reliable for a long time. It is as reliable as far as you trust VFP.
  • I use text only on my professional work. We had a problem with CodePages and a Apple with virtual Windows fiddling with Spanish strings in the last year. That's all, and it took all those years to bubble up. The chance to loose data is less then corrupting a vcx. (10:0 by last year. Guess what I have done to rebuild the vcx.)
• git is no backup
• no backup - no mercy
• since it is very easy to follow the changes in the text files (in special, as recommended, if one split VCX's into single classes) through git, there is little chance to loose anything.
• If one not work on a straight forward branching as you showed at SWFox, sooner or later one MUST merge a file. This is only possible on text base. And then one MUST transform. Do not forget do rebuild. Better to be used to.
  • but then your you will not find the change, because of hundreds of changed binaries, possibly with real changes (adding the text files), possibly only the binaries
  • even if you mixed up the recreate. You are on git. Create a branch on the fly, commit. diff, merge, bring it into one, recreate. What could go wrong? All is there, all is stored in git, all changes are visible.
  • If one is not ready to merge, on should use VSS checkin-check out. git is the wrong tool then. Might work in own organization, but this is for VFPX - independent developers on the same file.
• if two sides change the the same binary, you end up with unsolvable conflicts.
• It is enough if two compile, commit and push and you have any binary source mixed up. The text files will not change, but how to see this on hundreds of files, or tell git what to do? This is just a reason to pick the wrong file, because one must do this manually.

With binary sources one can not work like git, one end up VSS-like file picking. git is not about a single file, it's about the project. The idea is, tjat a commit stores the whole state of the project. It's abused a lot by folks that have not done the work to relearn, I know there are a lot of those let-us-pick-a-file GUIs out, but this is not the idea. git commit -a is all what one should need, if nothing is added, else git add.;git commit. One line. Only it will fail on every open vcx, and on the pjx.

• The single-file approach is cherrypicking. Waste of time.
• One should commit if a step is done, not after several. Possibly after each save. So what is changed in a step belongs to the step, why pick files?
• Text base commits are tiny. But binary go as full. Every [..] time.

It might be a different work flow. I created a tool that simply runs through my folder, picks any pjx or pj2 and recreates it as a whole.
Bin2Text, available under VFPX. Naturally, not intended for cherrypicking. Just go ahead. Transform, commit.

If one is using git as one should, commit often and in total, there is practical no chance to loose data. It is belt-and-braces anyway. The binaries are a parachute (set) on top. As clumsy, as useless.

In special

• FoxBin2Prg does not much with binary sources, except some tables and additional projects nobody need to open. Anyway, they are a pain. The pjx kills my workflow.

Other projects

Like for FoxBin2Prg for VFPX based projects I maintain there will be a Create / Recreate prg available for convenience. It as a real pain to my workflow to have the binaries in the repo, so that I count the inconvenience of those recreating little to mine. If I work, I have the problem on commiting once in a kilosecond. I branch and merge and compare often. I rely on the trace in git. All this is impossible. For others it's once per a pull issue.

[lscheffler]

@RickBorup

1. Anyone who has already cloned the repo and subsequently pulls from the remote to get the latest will quite reasonably assume they have the latest binaries. If the binaries are not in the remote repo, an extra step in required to regenerate them from the ??2 file, and this can easily be overlooked.

I'm a bit sorry, but I'm just provoked to say: Failed, repeat. ;)

It's very simple

• local once pulled the repo with a certain file in it
• remote removes the file out of the repo
• local pulls again
• the working directory becomes the state of the branch pulled
• a file that was under git supervision is deleted -> it is deleted on working directory too
• iow, for this special case, no vcx, no scx, no pjx I promise, local will notify.
• now local either look up the changelog file. or the readme, or asks questions. All of this should help to solve the "problem".

On subsequent pulls, the binaries will not be touched, they are ignored by git. There might be a little oddity if on switches between the both states, but nothing a recreate will not fix.

And if local groked to create the text versions, he will learn to do so for the binaries. It's just a question of the right tool.

Nothing is fool prove. I run myself into non-recreated-binaries. It was never a big deal. Normally I was testing something new, and it was not working. Simply looking up the code showed that the change was not where it belonged, so I gave myself a (－‸ლ) ...

Don't do this: If you are really afraid of not recreating. git offers something named hooks. You can tell git to run a command after some operations. So for anything that moves HEAD like checkout (switch) and merge, run FoxBin2Prg Text2Bin Folder. It's overkill, and a waste of time, if you simply like to look up something you will be bored to dead, but you couldn't miss recreate. It might be wise to secure that your changes are commited, else you might really loose work there is a pre-commit hook, possibly you should use this to create the binaries in general. I vote for Bin2Text, it's faster then plain FoxBin2Prg ...
I choose not to do it - I recreate after I finish operation on git side.

And remember, even if you count something rubbish, commit. You can always move the branch around or reset. Or you have one awkward commit in the line. How cares? Manager? If the manger complains, it's time to move anyway. There is nothing good in a clean or tidy history. Lost code is.

[RickBorup]

Perhaps I am not seeing what you are trying to describe here, but this makes no sense to me. If I understand what you are saying, after you pull from the remote the binaries are removed from the local because they are not tracked in Git. So what happens the first time you want to work on a project? You must first regenerate the PJX/PJT binaries from the PJ2. Then when you open the project it detects that none of the binaries are present and shows an error for each one. And if you want to work on one you must also regenerate it. From: Lutz Scheffler ***@***.***> Sent: Monday, December 4, 2023 1:55 PM To: fdbozzo/foxbin2prg ***@***.***> Cc: Rick Borup ***@***.***>; Mention ***@***.***> Subject: Re: [fdbozzo/foxbin2prg] Binary sources (Discussion #103) @RickBorup <https://github.com/RickBorup> 1. Anyone who has already cloned the repo and subsequently pulls from the remote to get the latest will quite reasonably assume they have the latest binaries. If the binaries are not in the remote repo, an extra step in required to regenerate them from the ??2 file, and this can easily be overlooked. I'm a bit sorry, but I'm just provoked to say: Failed, repeat. ;) It's very simple * local once pulled the repo with a certain file in it * remote removes the file out of the repo * local pulls again * the working directory becomes the state of the branch pulled * a file that was under git supervision is deleted -> it is deleted on working directory too * iow, for this special case, no vcx, no scx, no pjx I promise, local will notify. * now local either look up the changelog file. or the readme, or asks questions. All of this should help to solve the "problem". On subsequent pulls, the binaries will not be touched, they are ignored by git. There might be a little oddity if on switches between the both states, but nothing a recreate will not fix. And if local groked to create the text versions, he will learn to do so for the binaries. It's just a question of the right tool. Nothing is fool prove. I run myself into non-recreated-binaries. It was never a big deal. Normally I was testing something new, and it was not working. Simply looking up the code showed that the change was not where it belonged, so I gave myself a (－‸ლ) ... Don't do this: If you are really afraid of not recreating. git offers something named hooks. You can tell git to run a command after some operations. So for anything that moves HEAD like checkout (switch) and merge, run FoxBin2Prg Text2Bin Folder. It's overkill, and a waste of time, if you simply like to look up something you will be bored to dead, but you couldn't miss recreate. It might be wise to secure that your changes are commited, else you might really loose work there is a pre-commit hook, possibly you should use this to create the binaries in general. I vote for Bin2Text, it's faster then plain FoxBin2Prg ... I choose not to do it - I recreate after I finish operation on git side. And remember, even if you count something rubbish, commit. You can always move the branch around or reset. Or you have one awkward commit in the line. How cares? Manager? If the manger complains, it's time to move anyway. There is nothing good in a clean or tidy history. Lost code is. — Reply to this email directly, view it on GitHub <#103 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACMHRZOMBLCOETJ3WYYAVDLYHYTARAVCNFSM6AAAAABAF4YWDKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TONJXGA3DS> . You are receiving this because you were mentioned. <https://github.com/notifications/beacon/ACMHRZJVDOV74Y7FEVURI5TYHYTARA5CNFSM6AAAAABAF4YWDKWGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAOZOQ2.gif> Message ID: ***@***.*** ***@***.***> >

…

-- This email has been checked for viruses by Avast antivirus software. www.avast.com

[lscheffler]

Hi @RickBorup

Perhaps I am not seeing what you are trying to describe here, but this makes no sense to me.

If I understand what you are saying, after you pull from the remote the binaries are removed from the local because they are not tracked in Git. So what happens the first time you want to work on a project? You must first regenerate the PJX/PJT binaries from the PJ2. Then when you open the project it detects that none of the binaries are present and shows an error for each one. And if you want to work on one you must also regenerate it.

You need to step away from cherrypicking. Either all of the pj2 or whole folder

Create all binaries of a pj2:

DO foxbin2prg WITH "project.pj2","*"

This will regenerate the pjx and all files.
Syntax is:

• DO foxbin2prg WITH "project.pj2" just create the pjx
• DO foxbin2prg WITH "project.pj2","*" create pjx and files
• DO foxbin2prg WITH "project.pj2","*-" create files only
• DO foxbin2prg WITH "project.pjx",... create text-files, as above

To create all binaries in a folder structure

CD into_your_folder
DO FoxBin2Prg.PRG WITH JUSTPATH(FULLPATH("","")),"Prg2Bin"

If you use the file provided, all will be generated. Give it a try. Clone from https://github.com/lscheffler/foxbin2prg, follow the instructions on readme.md. This is the version that will go live in short time. (For Thor, this all does not matter, it runs out of the zip anyway.)

Update:
I really like you to test it. I'm just alone with my ideas ...

[RickBorup]

OK, I think I see where you are coming from with this. The advantage is not needing to include the binaries in the repository. The trade-off is that you must regenerate the binaries every time you need to make modifications, but as you have pointed out this can be done in one step for a project or a folder. My concern is that the risk of an error occurring when regenerating the binaries for a particular file is small but not zero, so to be prudent one would be well advised to keep a copy of the current binaries somewhere else as a backup. From: Lutz Scheffler ***@***.***> Sent: Tuesday, December 5, 2023 9:55 AM To: fdbozzo/foxbin2prg ***@***.***> Cc: Rick Borup ***@***.***>; Mention ***@***.***> Subject: Re: [fdbozzo/foxbin2prg] Binary sources (Discussion #103) Hi @RickBorup <https://github.com/RickBorup> Perhaps I am not seeing what you are trying to describe here, but this makes no sense to me. If I understand what you are saying, after you pull from the remote the binaries are removed from the local because they are not tracked in Git. So what happens the first time you want to work on a project? You must first regenerate the PJX/PJT binaries from the PJ2. Then when you open the project it detects that none of the binaries are present and shows an error for each one. And if you want to work on one you must also regenerate it. You need to step away from cherrypicking. Either all of the pj2 or whole folder Create all binaries of a pj2: DO foxbin2prg WITH "project.pj2","*" This will regenerate the pjx and all files. Syntax is <https://github.com/fdbozzo/foxbin2prg/blob/master/docs/FoxBin2Prg_Run.md#usage-1> : * DO foxbin2prg WITH "project.pj2" just create the pjx * DO foxbin2prg WITH "project.pj2","*" create pjx and files * DO foxbin2prg WITH "project.pj2","*-" create files only * DO foxbin2prg WITH "project.pjx",... create text-files, as above To create all binaries in a folder structure CD into_your_folder DO FoxBin2Prg.PRG WITH JUSTPATH(FULLPATH("","")),"Prg2Bin" If you use the file provided, all will be generated. Give it a try. Clone from https://github.com/lscheffler/foxbin2prg, follow the instructions on readme.md <https://github.com/lscheffler/foxbin2prg#regenerate-after-download--clone--pull> . This is the version that will go live in short time. (For Thor, this all does not matter, it runs out of the zip anyway.) — Reply to this email directly, view it on GitHub <#103 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACMHRZNQCT5ATUM2ASZXFLDYH47URAVCNFSM6AAAAABAF4YWDKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TONRWGI2DE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

…

-- This email has been checked for viruses by Avast antivirus software. www.avast.com

[lscheffler]

@RickBorup I understand the feeling. I found myself awkward then as well. Only It works for nearly a decade now. I count the missing backup-files for VFPs normal development more odd. And I know people that count vcx's and the like to high a risk in the 2010s. To me, text based git reduces the chance to loose code.

A rolling backup (using a tool named SecondCopy) I do since to 90s or so. I remember it if I need it, but I do not without. Before git it was the time machine, now only if I miss to commit for a while.

For all I remember, FoxBin2Prg owns options to backup by itself. Look up the options for config file in the documentation.

The main advantage of non binaries is to see what a commit is doing - or what is touched by a pull request.
Secondary that it eliminates the cherrypicking for such circumstances. I count the risk to miss to commit something and loose it at all higher then the chance FoxBin2Prg will fail to regenerate. Several order of magnitude higher.

And if. In worst case. The text file is still there. Might be hell, but nothing is lost, it could be done by hand. Only don't store whole vcx to a single vc2.
And, to be true, on some comp I use secondary, an old version of the binary will exist, and I will use the text file to do the change. Comparing on FoxBin2Prg level, naturally.