From candyshop999 at gmail.com Fri Jan 4 12:50:47 2008 From: candyshop999 at gmail.com (Super Star) Date: Fri, 4 Jan 2008 12:50:47 +0200 Subject: United Built Homes, SEO for Real Estate Companies Message-ID: United Built Homes, SEO for Real Estate Companies Definitely, every real estate companies want to have a successful business by gaining popularity, earning a lot and leading its competitors and so on. In order to have a successful real estate company, one should undergo internet marketing that can aid you to success. One of the best internet marketing is SEO or search engine optimization. SEO can increase the traffic of a real estate company?s website. It is an important tool that can make your real estate company be popular and can gain plenty of profits. With the aid of SEO, you can have plenty of visitors to your website which can be your customers in time. Most people use the internet through the search engines for information and even for searching products and services, so as a real estate company, you can have bigger chance to have a lot of visitors to your website. So you will be able to attract all of these visitors in order to gain plenty of customers. Internet users get information through the search engines by just typing the words that they are searching for, the search engines will give about thousand of results, but internet users open those websites on the top rank only, since they believe that those websites are the reliable and dependable ones. As a real estate company, you definitely want to be on the top rank of the search engines. With the help of SEO, your business will absolutely be on the top rank of search engines results like with Google, Yahoo and MSN. If your real estate company make use of search engine optimization, your business? website can obtain traffic and can be on the top rank of the search engines and your company will be on the lead with your competitors. Yahoo, Google and MSN, and other search engines, use algorithms in order to give information to internet users who are looking for information online. SEO use a lot of methods that can make a real estate company?s website be on the top rank of search engines listings continually and this can happen with the use of keyword rich articles. So for you to continually be on the top rank, you have to do and supply articles that are keyword rich. Since these keyword rich articles are useful not only to internet users but with the search engines as well. With the aid of SEO Company, a real estate company can obtain its goals such as being on the top rank of the search engines, be ahead of your competitors, gain visibility and popularity and of course to gain plenty of profits. But of course, you have to find the right SEO Company that can help you out and work for your real estate company?s success. The right SEO Company uses methods that can help you gain the goals you want. The right SEO Company can supply you with keyword rich articles. The best way to find the right SEO Company is by searching it through the internet. As a real estate company, if you are into Miami real estate, just simply type these words to the major search engines and find the websites who are on the top rank and look for the SEO Company who made them on that place. Then contact the SEO Company, this Company can definitely aid your real estate company to success. Eliza Maledevic Ayson fromJump2Top - SEO Company SEO Company -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mx.gw.com/pipermail/file/attachments/20080104/f785a664/attachment.html From candyshop999 at gmail.com Fri Jan 4 12:32:30 2008 From: candyshop999 at gmail.com (Super Star) Date: Fri, 4 Jan 2008 12:32:30 +0200 Subject: Martin Pelmore, Being Careful With Credit Cards Message-ID: Martin Pelmore, Being Careful With Credit Cards When it comes to getting equipped with a credit card, many people are aware of the advantages and disadvantages, but few are aware of how to ensure that they are getting the best deal through being careful about the credit card provider they use, and ensuring they have read the terms and conditions, and are therefore aware of all the cost centers that will affect them during their contract. Watch out for: Annual Fees ? Not all credit card providers will charge an annual fee, but by the same token not all those who do charge an annual fee will necessarily be the least cost-effective. Keep in mind that different credit card providers will charge in different ways, and being away of the ways you are charged means you get the best deal, and can budget for these expenses. Late Payment Charges ? Should your credit card payments be later than expected then as well as the interest still being due you will have to incur a late payment fee. This is justified by the credit card providers through the consideration that it could cause cash flow problems, and may mean that their financial planning could be off-set or not as projected. Late charges should be a consideration when costing a credit card; a great way to avoid this is to set up a direct debit so your payment is taken directly out of your bank account. Exceeding Your Limit ? When you exceed your credit limit the likelihood is that you will be charged, unless you make prior arrangements with your credit card provider. You should be aware of what these charges are before you enter into a contract with a credit card provider. To avoid these costs, many choose to take out another credit card to finance clearing the limit, whilst others may look towards short-term solutions like an overdraft or long-term solutions like a bank loan. Please be careful not to spend money on a credit card unless you can afford it. Avoid: Small Companies ? Avoiding small companies can ensure that you do not come across problems that would not face well-known, reputable house-hold names. Should a company be a public limited company/listed company or be owned by such then the inherent risk of unfair contract terms and a shift away from best-practice are significantly reduced by PR factors and media pressures/influences. High APR/0% initial APR ? Choosing to contract with a credit card company that offers a great introduction rate and a higher rate in the long term is relying on your lack of financial control to make money. If you are not in total control of your finances and ultimately earn less than you make then please avoid these at all costs. This article is written by Devin Gilliland publisher for credit-wisdom and jetclient.co -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mx.gw.com/pipermail/file/attachments/20080104/cfe7c2a3/attachment.html From candyshop999 at gmail.com Fri Jan 4 14:38:15 2008 From: candyshop999 at gmail.com (Super Star) Date: Fri, 4 Jan 2008 14:38:15 +0200 Subject: Palm Vacations, A Hawaii Vacation for the Entire Family Message-ID: Palm Vacations, A Hawaii Vacation for the Entire Family Ask a child over the age of 10 where they?d like to go on vacation and don?t be shocked when they tell you they want to go to Hawaii. After all, they?ve likely seen a lot of fun in Hawaiian commercials. They know there are fire-lit luau parties and hula dancers. They?ve seen the images and now, they want to be part of those scenes. They want to live the Hawaii-vacation dream! Hawaii vacations are the perfect family getaway. First, you can expect children and adults of all ages to find something to do anytime day or night. Depending on which island you visit and of course, the resort you choose, you should be able to find one that will appeal to everyone in your party. The younger generation, especially the teenagers will enjoy the surfing as well as the other beach activities found in Hawaii. Trips to Hawaii would never be complete unless a lot of time is spent on the gorgeous unspoiled beaches of many of the islands. After all, the surfing is best in many areas of Hawaii and teenagers do understand this which is why so many of them fall in love with the idea of a Hawaiian vacation. Golfing in Hawaii is another family activity for you to enjoy. Cheap Hawaiian vacation packages can be found with golf included in many of them. Golf is such a big deal on the Hawaiian Islands that there are over 80 golf courses spread out for your enjoyment. Sneak away for your tee-times or take the entire family. Most golf courses are very family-friendly. One of the best ways to see Hawaii as a family is to find a cruise setting sail to the various Islands. If you can find a large cruise liner sailing to the islands, you can generally see and do more in a short period of time. Of course, it is the ideal way to vacation as a group because so many shore excursions are planned taking into consideration the element of family! Families that visit Hawaii love the local celebrations which ultimately allow for a feast of fresh fruits and fire-grilled proteins for a Hawaiian buffet dinner and show. Many of the resorts offer various luaus complete with the beautiful hula girls and floral lei which is their customary flower necklace that the locals traditionally give to visitors. Of course many of the resorts have unbelievable pool areas with fantastic water slides, poolside service, and other amenities. Some of these Hawaiian resorts even offer games, crafts, and day camps for families with children. Of course, there?s always the family activity of snorkeling which is a fun time for everyone in the family. After all, Hawaii is perfect for snorkeling and equipment can even be rented for a small fee. Yes, Hawaii is a great place for families. It is inexpensive if you plan ahead and the entertainment is unmatched. Just choose your Island and book your vacation or better yet, book your cruise and see all of the beautiful Hawaiian Islands! -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mx.gw.com/pipermail/file/attachments/20080104/3551afe0/attachment.html From candyshop999 at gmail.com Fri Jan 4 14:30:55 2008 From: candyshop999 at gmail.com (Super Star) Date: Fri, 4 Jan 2008 14:30:55 +0200 Subject: John Rosatti, Tips On Caring For Your Cut Flowers Message-ID: John Rosatti, Tips On Caring For Your Cut Flowers Imagine it's a special occasion and you've just received a glorious bunch of roses. You put them in your best vase and you stand back to admire them. You'd love to make them last as long as you possibly can, but other than keeping them out of direct sunlight, you're not sure what to do. Here are some tips on how to extend the shelf life of your roses: 1. Remove the roses from the vase and put them in a temporary container. 2. Clean the vase by washing it thoroughly in water with household bleach. 3. Fill the vase with fresh water and add a half teaspoon of household bleach for each litre (16 fluid ounces) of water. 4. Remove your roses from their temporary container and lay them flat on a counter top. 5. Remove any leaves which will show below the waterline in the vase. 6. Cut about 20 mm (3/4 inch) from the base of each stem immediately before placing it in the vase. This removes any air bubbles. 7. Don't place roses in drafts or direct sunlight, or near fruit, heaters, TVs or refrigerators. 8. Change the water daily to extend the vase life of your roses. Freshly picked roses can last up to two weeks with a little extra care. Enjoy! Gerry Belvedere is a former artist who now runs her own online florist service at . Rosaflora delivers flowers Australia wide and overseas and offers tips on getting the most out of your cut flowers. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mx.gw.com/pipermail/file/attachments/20080104/65f1a519/attachment.html From candyshop999 at gmail.com Fri Jan 4 15:54:42 2008 From: candyshop999 at gmail.com (Super Star) Date: Fri, 4 Jan 2008 15:54:42 +0200 Subject: Douglas Della Toffalo, How Can Sports Psychology Help Athletes? Message-ID: Douglas Della Toffalo, How Can Sports Psychology Help Athletes? Athletes often ask me this question: "How do I know when a poor performance or error is a physical or a mental problem?" From my experience, here are a few hints that it may be a mental breakdown: 1. You perform much better in practice than during competition 2. You have a tough time performing well when others are watching you 3. You maintain many doubts about your sport before or during games 4. You feel anxious or scared when you perform in competition 5. You are not sure why you play your sport or what motivates you 6. You only participate in sports to feel better about yourself as a person 7. You lose focus or have mental lapses during critical times of the game 8. You can't perform the way you did pre-injury, but are physically 100% 9. Everything is fine, but you just want to improve your mental attitude Sports psychology sometimes get a bad reputation because of the association of psychology with pathology. That is why I prefer to call what I do mental game coaching or mental training, which athletes understand. Mental game coaching is for athletes who want to improve upon their current performance and take their games to the next level with the help of a mental coach like myself. But most athletes, unfortunately, seek out my services because of an particular performance barrier or decrease in performance. As a mental game coach, I often become the last resort after athletes have tried other means to get beyond performance slumps. I wish it wasn't this way, but athletes wait until some needs to be "fixed" and they have exhausted all other resources before they commit to mental game coaching. How can sport psychology help you perform better? Here is the most obvious list: 1. Improve focus and deal with distractions. 2. Grow confidence in athletes who have many doubts. 3. Develop coping skills to deal with setbacks and errors. 4. Find the right zone of intensity for your sport. 5. Help teams develop communication skills and cohesion. 6. To instill a healthy belief system and weed out irrational thoughts. 7. Improve and balance motivation for optimal performance. 8. Get back into competition after an injury is healed fully. 9. To develop game-specific strategies and game plans. 10. To identify and enter the "zone" more often. Sport psychology may not be for every single athlete. Not everyone involved in sport wants to "improve performance." Sport psychology is probably not for recreation athletes who just like the social component of sport. Most of my students are committed to excellence and seeing how far they can go. They love competition and testing themselves against the best in their sport. They want every possible advantage they can learn including the mental edge over the competition. Dr. Patrick J. Cohn is a leading mental game coach who consults with Tour Pros and amateurs. He is the author of Going Low, Peak Performance Golf, The Mental Game of Golf and The Mental Art of Putting. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mx.gw.com/pipermail/file/attachments/20080104/384426e5/attachment-0001.html From christos at zoulas.com Tue Jan 8 01:32:31 2008 From: christos at zoulas.com (Christos Zoulas) Date: Mon, 7 Jan 2008 18:32:31 -0500 Subject: Magic database submission: Mercurial changeset bundle In-Reply-To: <4782B367.7060804@sun.com> from Jesse Glick (Jan 7, 6:19pm) Message-ID: <20080107233231.B262D56407@rebar.astron.com> On Jan 7, 6:19pm, Jesse.Glick at sun.com (Jesse Glick) wrote: -- Subject: Magic database submission: Mercurial changeset bundle | Tested in /etc/magic on some examples including the attached file. Thanks, I have added it. Best, christos From program.spe at home.pl Mon Jan 14 13:40:50 2008 From: program.spe at home.pl (Krzysztof =?UTF-8?Q?=C5=BBelechowski?=) Date: Mon, 14 Jan 2008 12:40:50 +0100 Subject: file does not recognize Visual FoxPro table files Message-ID: <1200310851.6524.9.camel@a1dmin.vola.spe.com.pl> Microsoft Visual FoxPro table files are a variant of dBase table files. They are documented by Microsoft at . While the file utility correctly recognizes dBase table files, it fails to recognize the FoxPro variant. Example: ? file 'FASTE.DBF' > FASTE.DBF: DBase 3 data file (14 records) ? file 'aaa.DBF' > aaa.DBF: data ? python 'doit.py' 'FASTE.DBF' > FoxBASE+ Latest update 1995-03-16 14 records flags: 0 Code page: 0 > ? python 'doit.py' 'aaa.DBF' > Visual FoxPro Latest update 2003-10-03 1 records flags: 0 Code page: 200 > where 'doit.py' is a custom script to parse the header along the lines of the documentation. From colin at e-e.com Wed Jan 23 20:40:17 2008 From: colin at e-e.com (Colin Bartolome) Date: Wed, 23 Jan 2008 10:40:17 -0800 Subject: MIME-type magic hierarcy Message-ID: <47978A11.3010907@e-e.com> I've got a similar problem. With rules like this: 0 string DOC >144 string JDASH application/x-ichitaro4 I can make a test file that produces this output: [user at host ~]$ file -im magic doc doc: application/x-ichitaro4 [user at host ~]$ Then I can change the file so the DOC rule matches but the JDASH rules doesn't. Then I get this output: [user at host ~]$ file -im magic doc doc: [user at host ~]$ Also note what happens when I add the -k flag, as it matches all the DOC rules but none of the continuations: [user at host ~]$ file -rkim magic doc doc: - - [user at host ~]$ I think there should be a flag where if a rule's continuation doesn't match, nothing in the entire tree is printed and it just moves on. This file should just wind up as text/plain. Please let me know if there's already an option for this. - Colin Bartolome From christos at zoulas.com Wed Jan 23 22:43:19 2008 From: christos at zoulas.com (Christos Zoulas) Date: Wed, 23 Jan 2008 15:43:19 -0500 Subject: MIME-type magic hierarcy In-Reply-To: <47978A11.3010907@e-e.com> from Colin Bartolome (Jan 23, 10:40am) Message-ID: <20080123204319.737E456407@rebar.astron.com> On Jan 23, 10:40am, colin at e-e.com (Colin Bartolome) wrote: -- Subject: MIME-type magic hierarcy | I've got a similar problem. With rules like this: | | 0 string DOC | >144 string JDASH application/x-ichitaro4 | | I can make a test file that produces this output: | | [user at host ~]$ file -im magic doc | doc: application/x-ichitaro4 | [user at host ~]$ | | Then I can change the file so the DOC rule matches but the JDASH rules | doesn't. Then I get this output: | | [user at host ~]$ file -im magic doc | doc: | [user at host ~]$ | | Also note what happens when I add the -k flag, as it matches all the DOC | rules but none of the continuations: | | [user at host ~]$ file -rkim magic doc | doc: | - | - | [user at host ~]$ | | I think there should be a flag where if a rule's continuation doesn't | match, nothing in the entire tree is printed and it just moves on. This | file should just wind up as text/plain. | | Please let me know if there's already an option for this. | What version of file are you using? I believe this is fixed in 4.23. christos From colin at e-e.com Wed Jan 23 23:00:28 2008 From: colin at e-e.com (Colin Bartolome) Date: Wed, 23 Jan 2008 13:00:28 -0800 Subject: MIME-type magic hierarcy References: 47978A11.3010907@e-e.com Message-ID: <4797AAEC.8090402@e-e.com> Looks like I was using file-4.12 in /usr/bin/file instead of file-4.23 in my home directory. Whoops! Time to talk to the sysadmin! Thanks. From program.spe at home.pl Wed Feb 6 18:41:41 2008 From: program.spe at home.pl (Krzysztof =?UTF-8?Q?=C5=BBelechowski?=) Date: Wed, 06 Feb 2008 17:41:41 +0100 Subject: GnuPG encrypted files are not recognized Message-ID: <1202316102.6645.49.camel@a1dmin.vola.spe.com.pl> When I ask what a GnuPG-encrypted file is, it says "data". Can that be changed to say "PGP-encrypted data"? Chris From vapier at gentoo.org Thu Feb 14 21:31:55 2008 From: vapier at gentoo.org (Mike Frysinger) Date: Thu, 14 Feb 2008 14:31:55 -0500 Subject: file mailing list info not found in tarball anywhere Message-ID: <200802141431.55929.vapier@gentoo.org> could we get information about the file at mx.gw.com mailing list added to the tarball and associated files ? for example, i'd add it to: - README - MAINT - doc/*.man (a new "BUG REPORTS" section) -mike -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 827 bytes Desc: This is a digitally signed message part. Url : http://mx.gw.com/pipermail/file/attachments/20080214/34a370c7/attachment.bin From vapier at gentoo.org Thu Feb 14 21:34:11 2008 From: vapier at gentoo.org (Mike Frysinger) Date: Thu, 14 Feb 2008 14:34:11 -0500 Subject: `file --mime` outputs extraneous info when run on ELFs Message-ID: <200802141434.12001.vapier@gentoo.org> normally the --mime option outputs just the mime type and nothing else, but when it is run on an ELF, we get a lot of extra junk in there: $ file --mime /bin/bash /bin/bash: application/x-executable, for GNU/Linux 2.6.9, dynamically linked (uses shared libs), stripped clearly we dont want anything after "application/x-executable" :) -mike -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 827 bytes Desc: This is a digitally signed message part. Url : http://mx.gw.com/pipermail/file/attachments/20080214/eafdb9ca/attachment.bin From christos at zoulas.com Thu Feb 14 22:22:42 2008 From: christos at zoulas.com (Christos Zoulas) Date: Thu, 14 Feb 2008 15:22:42 -0500 Subject: file mailing list info not found in tarball anywhere In-Reply-To: <200802141431.55929.vapier@gentoo.org> from Mike Frysinger (Feb 14, 2:31pm) Message-ID: <20080214202242.AE8E256534@rebar.astron.com> On Feb 14, 2:31pm, vapier at gentoo.org (Mike Frysinger) wrote: -- Subject: file mailing list info not found in tarball anywhere | could we get information about the file at mx.gw.com mailing list added to the | tarball and associated files ? for example, i'd add it to: | - README | - MAINT | - doc/*.man (a new "BUG REPORTS" section) | mike Added to README. christos From christos at zoulas.com Thu Feb 14 22:23:07 2008 From: christos at zoulas.com (Christos Zoulas) Date: Thu, 14 Feb 2008 15:23:07 -0500 Subject: `file --mime` outputs extraneous info when run on ELFs In-Reply-To: <200802141434.12001.vapier@gentoo.org> from Mike Frysinger (Feb 14, 2:34pm) Message-ID: <20080214202307.A19E956542@rebar.astron.com> On Feb 14, 2:34pm, vapier at gentoo.org (Mike Frysinger) wrote: -- Subject: `file --mime` outputs extraneous info when run on ELFs | normally the --mime option outputs just the mime type and nothing else, but | when it is run on an ELF, we get a lot of extra junk in there: | | $ file --mime /bin/bash | /bin/bash: application/x-executable, for GNU/Linux 2.6.9, dynamically linked | (uses shared libs), stripped | | clearly we dont want anything after "application/x-executable" :) | mike fixed, christos From christos at zoulas.com Mon Feb 18 17:25:21 2008 From: christos at zoulas.com (Christos Zoulas) Date: Mon, 18 Feb 2008 10:25:21 -0500 Subject: OS/2 apptype file feature Message-ID: <20080218152521.97EE256534@rebar.astron.com> Does anyone use OS/2 anymore? I am asking because I would like to remove the OS/2 specific code from file that uses DosQueryAppType(). Is that ok? christos From faith at cs.unc.edu Sat Feb 23 17:14:33 2008 From: faith at cs.unc.edu (Rik Faith) Date: Sat, 23 Feb 2008 10:14:33 -0500 Subject: ACECAD DigiMemo file Message-ID: <18368.14425.439322.39019@gargle.gargle.HOWL> The file(1) command currently thinks the attached file is "data", but it was generated from a DigiMemo L2 / A402 digital note pad. I think almost all the ACECAD products generate a similar file format, but I have no evidence of that. My file is from a DigiMemo L2 / A402. Here's a Magdir/acecad file that seems to work well: # ACECAD DigiMemo 0 string ACECAD_DIGIMEMO ACECAD Digital Handwriting File, >0x20 byte x version %2x, >0x21 leshort x width=%d, >0x23 leshort x height=%d >0x25 byte =0 A5, >0x25 byte =1 A4, >0x25 byte =2 Letter, >0x25 byte =8 B5, >0x25 byte =9 B4, -------------- next part -------------- A non-text attachment was scrubbed... Name: pglt_001.dhw Type: application/octet-stream Size: 1644 bytes Desc: not available Url : http://mx.gw.com/pipermail/file/attachments/20080223/99a48f86/attachment.obj From gjalves at gjalves.com.br Fri Mar 7 15:55:42 2008 From: gjalves at gjalves.com.br (Gustavo Alves) Date: Fri, 7 Mar 2008 10:55:42 -0300 Subject: Cytovision cytogenetic analysis files Message-ID: <69e11d1f0803070555t339c7bat338ffd71c875f88@mail.gmail.com> This is the magic for some Applied Images Cytovision files. This application is used on cytogenetic image analysis. # Applied Images - Image files from Cytovision # Gustavo Junior Alves 0 string \xce\xda\xde\xfa Cytovision Metaphases file 0 string \xed\xad\xef\xac Cytovision Karyotype file 0 string \x0b\x00\x03\x00 Cytovision FISH Probe file 0 string \xed\xfe\xda\xbe Cytovision FLEX file 0 string \xed\xab\xed\xfe Cytovision FLEX file 0 string \xad\xfd\xea\xad Cytovision RATS file -- Gustavo Junior Alves GJAlves Tecnologia Tel: +55 19 9223-0500 From christos at zoulas.com Fri Mar 7 17:01:29 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 7 Mar 2008 10:01:29 -0500 Subject: Cytovision cytogenetic analysis files In-Reply-To: <69e11d1f0803070555t339c7bat338ffd71c875f88@mail.gmail.com> from "Gustavo Alves" (Mar 7, 10:55am) Message-ID: <20080307150129.CB72056534@rebar.astron.com> On Mar 7, 10:55am, gjalves at gjalves.com.br ("Gustavo Alves") wrote: -- Subject: Cytovision cytogenetic analysis files | This is the magic for some Applied Images Cytovision files. This | application is used on cytogenetic image analysis. | | # Applied Images - Image files from Cytovision | # Gustavo Junior Alves | 0 string \xce\xda\xde\xfa Cytovision Metaphases file | 0 string \xed\xad\xef\xac Cytovision Karyotype file | 0 string \x0b\x00\x03\x00 Cytovision FISH Probe file | 0 string \xed\xfe\xda\xbe Cytovision FLEX file | 0 string \xed\xab\xed\xfe Cytovision FLEX file | 0 string \xad\xfd\xea\xad Cytovision RATS file | Added, thanks! christos From vapier at gentoo.org Sat Mar 15 05:07:51 2008 From: vapier at gentoo.org (Mike Frysinger) Date: Fri, 14 Mar 2008 23:07:51 -0400 Subject: [patch] pull out Blackfin ELF headers Message-ID: <200803142307.52233.vapier@gentoo.org> this patch decodes a few important bits in the Blackfin ELF header -mike diff --git a/magic/Magdir/elf b/magic/Magdir/elf index 464b436..5442993 100644 --- a/magic/Magdir/elf +++ b/magic/Magdir/elf @@ -102,6 +102,8 @@ >>18 leshort 94 Tensilica Xtensa, >>18 leshort 97 NatSemi 32k, >>18 leshort 106 Analog Devices Blackfin, +>>>36 byte&0x1 0x1 PIC, +>>>36 byte&0x2 0x2 FDPIC, >>18 leshort 0x9026 Alpha (unofficial), >>20 lelong 0 invalid version >>20 lelong 1 version 1 From christos at zoulas.com Fri Mar 28 20:28:10 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 28 Mar 2008 14:28:10 -0400 Subject: file 4.24 is now available Message-ID: <20080328182810.292B056539@rebar.astron.com> Hello, I am pleased to announce file 4.24. There have been numerous bug fixes and enhancements: Reuben Thomas: - There is no separate mime file; the mime entries are now integrated in the regular magic file as !:mime annotations. - Separation of text and binary magic searches - UTF-8 BOM message clarification - Compilation of magic fragments from a directory - Unit test framework Bob Byrnes: - Fix ELF core detection on solaris Gavin Atkinson: - zero out utime/utimes structs Diego "Flameeyes" Petten: - reduce writable data As always file is at: ftp://ftp.astron.com/pub/file/file-4.24.tar.gz Enjoy, christos From christos at zoulas.com Fri Mar 28 21:54:34 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 28 Mar 2008 15:54:34 -0400 Subject: file 4.24 is now available In-Reply-To: <47ED433B.206@freethemallocs.com> from Jonathan Smith (Mar 28, 11:12am) Message-ID: <20080328195434.C711A56542@rebar.astron.com> On Mar 28, 11:12am, smithj at freethemallocs.com (Jonathan Smith) wrote: -- Subject: Re: file 4.24 is now available | Christos Zoulas wrote: | | I am pleased to announce file 4.24. | | Is it intentional that the LEGAL.NOTICE file is missing from this | release? It was present in 4.23. It moved to COPYING for more compliance with the GNU autoconf standards. christos From christos at zoulas.com Tue Apr 8 15:55:02 2008 From: christos at zoulas.com (Christos Zoulas) Date: Tue, 8 Apr 2008 08:55:02 -0400 Subject: Pack200 archive version fields In-Reply-To: <47FB0199.9010300@gmx.net> from Ivan Todoroski (Apr 8, 7:24am) Message-ID: <20080408125503.0C5325654E@rebar.astron.com> On Apr 8, 7:24am, grnch at gmx.net (Ivan Todoroski) wrote: -- Subject: Pack200 archive version fields | Hello Christos, | | Just like Java class files, the Pack200 archive format has major and minor | version fields. This patch prints them out, and also removes the redundant | Pack200 definition from Magdir/archive, as it appears Pack200 is now defined in | Magdir/cafebabe. | | Thank you for maintaining this tool, it's absolutely indispensable. | | | P.S. Apologies for sending this twice, the first time my email client mangled | the tabs in the inline patch, so I'm sending it as attachment now. Thanks, applied. christos From rob.bowers at onedingo.com Fri May 9 04:04:35 2008 From: rob.bowers at onedingo.com (Rob Bowers) Date: Fri, 09 May 2008 01:04:35 +0000 Subject: file 4.24 no longer identifies .ogg & .url correctly Message-ID: <200805090104.35777.rob.bowers@onedingo.com> Hi. I am using Arch Linux and file 4.24 no longer identifies .ogg & .url files. For .ogg: owner at amd1200:/common/netcast$ file bsdtalk148.ogg bsdtalk148.ogg: data owner at amd1200:/common/netcast$ hexdump -Cv bsdtalk148.ogg | head 00000000 4f 67 67 53 00 02 00 00 00 00 00 00 00 00 08 48 |OggS...........H| 00000010 db 44 00 00 00 00 cc 71 9c 61 01 1e 01 76 6f 72 |.D.....q.a...vor| 00000020 62 69 73 00 00 00 00 01 44 ac 00 00 00 00 00 00 |bis.....D.......| 00000030 00 77 01 00 00 00 00 00 b8 01 4f 67 67 53 00 00 |.w........OggS..| 00000040 00 00 00 00 00 00 00 00 08 48 db 44 01 00 00 00 |.........H.D....| 00000050 c2 f2 5e 03 10 2d ff ff ff ff ff ff ff ff ff ff |..^..-..........| 00000060 ff ff ff ff c9 03 76 6f 72 62 69 73 1d 00 00 00 |......vorbis....| 00000070 58 69 70 68 2e 4f 72 67 20 6c 69 62 56 6f 72 62 |Xiph.Org libVorb| 00000080 69 73 20 49 20 32 30 30 34 30 36 32 39 00 00 00 |is I 20040629...| 00000090 00 01 05 76 6f 72 62 69 73 29 42 43 56 01 00 08 |...vorbis)BCV...| For .url: (Microsoft Favorites/Web Shortcuts) owner at amd1200:/common/workspace$ cat > website.url [InternetShortcut] URL=http://www.server.com/ owner at amd1200:/common/workspace$ dos2unix --u2d website.url owner at amd1200:/common/workspace$ file website.url website.url: MS Windows 95 Internet shortcut text (URL=< >) URL=< > above contains nothing but it used to. I just subscribed to this list today and the mailing list archive ends on April 8, so I don't know if this has been posted already. Please respond directly as I have not received a subscription confirmation yet. Thanks. R. From christos at zoulas.com Sat May 10 18:40:24 2008 From: christos at zoulas.com (Christos Zoulas) Date: Sat, 10 May 2008 11:40:24 -0400 Subject: file 4.24 no longer identifies .ogg & .url correctly In-Reply-To: <200805090104.35777.rob.bowers@onedingo.com> from Rob Bowers (May 9, 1:04am) Message-ID: <20080510154024.A5D805654F@rebar.astron.com> On May 9, 1:04am, rob.bowers at onedingo.com (Rob Bowers) wrote: -- Subject: file 4.24 no longer identifies .ogg & .url correctly | Hi. | | I am using Arch Linux and file 4.24 no longer identifies .ogg & .url files. | | For .ogg: | | owner at amd1200:/common/netcast$ file bsdtalk148.ogg | bsdtalk148.ogg: data | owner at amd1200:/common/netcast$ hexdump -Cv bsdtalk148.ogg | head | 00000000 4f 67 67 53 00 02 00 00 00 00 00 00 00 00 08 48 |OggS...........H| | 00000010 db 44 00 00 00 00 cc 71 9c 61 01 1e 01 76 6f 72 |.D.....q.a...vor| | 00000020 62 69 73 00 00 00 00 01 44 ac 00 00 00 00 00 00 |bis.....D.......| | 00000030 00 77 01 00 00 00 00 00 b8 01 4f 67 67 53 00 00 |.w........OggS..| | 00000040 00 00 00 00 00 00 00 00 08 48 db 44 01 00 00 00 |.........H.D....| | 00000050 c2 f2 5e 03 10 2d ff ff ff ff ff ff ff ff ff ff |..^..-..........| | 00000060 ff ff ff ff c9 03 76 6f 72 62 69 73 1d 00 00 00 |......vorbis....| | 00000070 58 69 70 68 2e 4f 72 67 20 6c 69 62 56 6f 72 62 |Xiph.Org libVorb| | 00000080 69 73 20 49 20 32 30 30 34 30 36 32 39 00 00 00 |is I 20040629...| | 00000090 00 01 05 76 6f 72 62 69 73 29 42 43 56 01 00 08 |...vorbis)BCV...| | | For .url: (Microsoft Favorites/Web Shortcuts) | | owner at amd1200:/common/workspace$ cat > website.url | [InternetShortcut] | URL=http://www.server.com/ | owner at amd1200:/common/workspace$ dos2unix --u2d website.url | owner at amd1200:/common/workspace$ file website.url | website.url: MS Windows 95 Internet shortcut text (URL=< >) | | URL=< > above contains nothing but it used to. | | I just subscribed to this list today and the mailing list archive ends on April 8, so I don't know if this has been posted already. | | Please respond directly as I have not received a subscription confirmation yet. #0 search/1000 OggS Ogg data 0 string OggS Ogg data christos From mdorey at bluearc.com Fri May 16 01:43:07 2008 From: mdorey at bluearc.com (Martin Dorey) Date: Thu, 15 May 2008 15:43:07 -0700 Subject: distressingly obscure error message on failing to open a file Message-ID: This change in magic.c, introduced between 4.23 and 4.24, caused a backup job of ours to produce a large number of mysterious messages saying "couldn't open file". It took quite some digging to work out (from find /usr/lib -type f | xargs strings | grep) what was producing such a message. @@ -296,10 +297,12 @@ ?????????errno = 0; ?????????if ((fd = open(inname, flags)) < 0) { ?#ifdef __CYGWIN__ +????????????/* FIXME: Do this with EXEEXT from autotools */ ?????????????char *tmp = alloca(strlen(inname) + 5); ?????????????(void)strcat(strcpy(tmp, inname), ".exe"); ?????????????if ((fd = open(tmp, flags)) < 0) { ?#endif +????????????????fprintf(stderr, "couldn't open file\n"); ?????????????????if (info_from_stat(ms, sb.st_mode) == -1) ?????????????????????goto done; ?????????????????rv = 0; I wanted to submit a patch, so I changed the fprintf line to: fprintf(stderr, "couldn't open file `%s': %s\n", inname, strerror(errno)); But then I tripped over a problem introduced between the same versions in funcs.c. The file_printf(ms, f, va) line - the one with the unusual indentation - is passing a va_list to a printf-style function rather than a vprintf-style function. That crashes for me. @@ -108,21 +81,17 @@ file_error_core(struct magic_set *ms, int error, const char *f, va_list va, uint32_t lineno) { - size_t len; /* Only the first error is ok */ if (ms->haderr) return; - len = 0; if (lineno != 0) { - (void)snprintf(ms->o.buf, ms->o.size, "line %u: ", lineno); - len = strlen(ms->o.buf); - } - (void)vsnprintf(ms->o.buf + len, ms->o.size - len, f, va); - if (error > 0) { - len = strlen(ms->o.buf); - (void)snprintf(ms->o.buf + len, ms->o.size - len, " (%s)", - strerror(error)); - } + free(ms->o.buf); + ms->o.buf = NULL; + file_printf(ms, "line %u: ", lineno); + } + file_printf(ms, f, va); + if (error > 0) + file_printf(ms, " (%s)", strerror(error)); ms->haderr++; ms->error = error; } Perhaps file_printf should be renamed file_vprintf and given a va_list argument, something like this: --- funcs.c.old 2008-05-15 15:15:31.000000000 -0700 +++ funcs.c 2008-05-15 15:28:28.000000000 -0700 @@ -45,18 +45,15 @@ * Like printf, only we append to a buffer. */ protected int -file_printf(struct magic_set *ms, const char *fmt, ...) +file_vprintf(struct magic_set *ms, const char *fmt, va_list ap) { - va_list ap; size_t size; int len; char *buf, *newstr; - va_start(ap, fmt); len = vasprintf(&buf, fmt, ap); if (len < 0) goto out; - va_end(ap); if (ms->o.buf != NULL) { len = asprintf(&newstr, "%s%s", ms->o.buf, buf); @@ -73,6 +70,18 @@ return -1; } +protected int +file_printf(struct magic_set *ms, const char *fmt, ...) +{ + va_list ap; + int len; + + va_start(ap, fmt); + len = file_vprintf(ms, fmt, ap); + va_end(ap); + return len; +} + /* * error - print best error message possible */ @@ -89,7 +98,7 @@ ms->o.buf = NULL; file_printf(ms, "line %u: ", lineno); } - vprintf(f, va); + file_vprintf(ms, f, va); if (error > 0) file_printf(ms, " (%s)", strerror(error)); ms->haderr++; Make sense? From christos at zoulas.com Fri May 16 17:22:18 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 16 May 2008 10:22:18 -0400 Subject: distressingly obscure error message on failing to open a file In-Reply-To: from "Martin Dorey" (May 15, 3:43pm) Message-ID: <20080516142218.A85DA5654E@rebar.astron.com> On May 15, 3:43pm, mdorey at bluearc.com ("Martin Dorey") wrote: -- Subject: distressingly obscure error message on failing to open a file | I wanted to submit a patch, so I changed the fprintf line to: | | fprintf(stderr, "couldn't open file `%s': %s\n", inname, strerror(errno)); I just removed the printf, thanks. | But then I tripped over a problem introduced between the same versions in f= | uncs.c. The file_printf(ms, f, va) line - the one with the unusual indenta= | tion - is passing a va_list to a printf-style function rather than a vprint= | f-style function. That crashes for me. Thanks a lot, I applied your fix. christos From Mark.Martinec+amavis at ijs.si Thu May 22 21:32:36 2008 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Thu, 22 May 2008 20:32:36 +0200 Subject: file-4.24 no longer recognizes .Z files (classical Unix compress files) Message-ID: <200805222032.36068.Mark.Martinec+amavis@ijs.si> Christos, Tod A. Sandman has reported on the amavis-user mailing list that the file-4.24 no longer recognizes classical Unix compressed files (the .Z files). I checked today and can confirm the problem: $ /usr/bin/file *.lis* # older version of 'file' 9.lis.Z: compress'd data 9 bits 10.lis.Z: compress'd data 10 bits 11.lis.Z: compress'd data 11 bits 12.lis.Z: compress'd data 12 bits 13.lis.Z: compress'd data 13 bits 14.lis.Z: compress'd data 14 bits 15.lis.Z: compress'd data 15 bits 16.lis.Z: compress'd data 16 bits $ file --version # new one file-4.24 magic file from /usr/local/share/file/magic $ file *.lis* 9.lis.Z: data 10.lis.Z: data 11.lis.Z: data 12.lis.Z: data 13.lis.Z: data 14.lis.Z: data 15.lis.Z: data 16.lis.Z: data These test files were just some plain text files compressed with a Unix or Linux 'compress' command. The result is the same regardless of the value of option -b (bits, 9..16). It would be nice to get this fixed by the next version. (please drop my previous posting waiting for moderation, I used a wrong sending address). Regards Mark From christos at zoulas.com Thu May 22 22:03:30 2008 From: christos at zoulas.com (Christos Zoulas) Date: Thu, 22 May 2008 15:03:30 -0400 Subject: file-4.24 no longer recognizes .Z files (classical Unix compress files) In-Reply-To: <200805222032.36068.Mark.Martinec+amavis@ijs.si> from Mark Martinec (May 22, 8:32pm) Message-ID: <20080522190330.191A85654E@rebar.astron.com> On May 22, 8:32pm, Mark.Martinec+amavis at ijs.si (Mark Martinec) wrote: -- Subject: file-4.24 no longer recognizes .Z files (classical Unix compress | Christos, | | Tod A. Sandman has reported on the amavis-user mailing list that | the file-4.24 no longer recognizes classical Unix compressed files | (the .Z files). I checked today and can confirm the problem: | | $ /usr/bin/file *.lis* # older version of 'file' | 9.lis.Z: compress'd data 9 bits | 10.lis.Z: compress'd data 10 bits | 11.lis.Z: compress'd data 11 bits | 12.lis.Z: compress'd data 12 bits | 13.lis.Z: compress'd data 13 bits | 14.lis.Z: compress'd data 14 bits | 15.lis.Z: compress'd data 15 bits | 16.lis.Z: compress'd data 16 bits | | $ file --version # new one | file-4.24 | magic file from /usr/local/share/file/magic | $ file *.lis* | 9.lis.Z: data | 10.lis.Z: data | 11.lis.Z: data | 12.lis.Z: data | 13.lis.Z: data | 14.lis.Z: data | 15.lis.Z: data | 16.lis.Z: data | | These test files were just some plain text files compressed | with a Unix or Linux 'compress' command. The result is the | same regardless of the value of option -b (bits, 9..16). | | It would be nice to get this fixed by the next version. | | (please drop my previous posting waiting for moderation, | I used a wrong sending address). Fixed, thanks; it was broken by the following change; just patch -R it. christos Index: compress =================================================================== RCS file: /p/file/cvsroot/file/magic/Magdir/compress,v retrieving revision 1.30 retrieving revision 1.31 diff -u -r1.30 -r1.31 --- compress 4 Feb 2008 16:33:45 -0000 1.30 +++ compress 5 Feb 2008 01:59:08 -0000 1.31 @@ -9,8 +9,8 @@ # because it tries to uncompress it to figure out what's inside. # standard unix compress -0 string \037\235 compress'd data -!:mime application/x-compress +#0 string \037\235 compress'd data +#!:mime application/x-compress >2 byte&0x80 >0 block compressed >2 byte&0x1f x %d bits From Ben.Stanley at exemail.com.au Tue Jun 17 07:39:23 2008 From: Ben.Stanley at exemail.com.au (Ben Stanley) Date: Tue, 17 Jun 2008 14:39:23 +1000 Subject: file 4.21 problem with multi-channel WAV files Message-ID: <1213677563.28032.320.camel@localhost> Hi, I have Ubuntu 7.10, bundling file 4.21-1. I don't know if this has been fixed in newer versions, but I thought I would report it to make sure. I wasn't able to test the latest version 4.24. I downloaded a multi-channel wave file from http://www.lindberg.no/hires/2L50SACD_tr01_multi_48.wav (Beware - the file is 149Mb! But you only need the first little bit to test the file command.) (The file comes from the first musical album released in BluRay format, see [1]. Some tracks are available in various formats for free download, see [2].) When I run the file command, I get some strange output, particularly for the number of channels: $ file --version file-4.21 magic file from /etc/magic:/usr/share/file/magic $ file 2L50SACD_tr01_multi_48.wav 2L50SACD_tr01_multi_48.wav: RIFF (little-endian) data, WAVE audio, 2396 channels Reading the header in khexedit shows that this file has 6 channels only, but the file command is reporting an absurdly large number of channels, and fails to report the sampling rate. Some further output from other files downloaded from that site is shown below. $ file * 2L38SACD_tr1_DTS_44kHz.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, stereo 44100 Hz 2L38SACD_tr1_mch_48kHz.wav: RIFF (little-endian) data, WAVE audio, 7435 channels 2L38SACD_tr1_stereo_96.flac: FLAC audio bitstream data, 24 bit, stereo, 96 kHz, 54146863 samples 2L38SACD_tr1_stereo_96.wav: RIFF (little-endian) data, WAVE audio, 4957 channels 2L50SACD_tr01_multi_48.wav: RIFF (little-endian) data, WAVE audio, 2396 channels 2L50SACD_tr1_DTS.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, stereo 44100 Hz Beethoven_op111_sats1_multikanal.flac: FLAC audio bitstream data, 24 bit, 6 channels, 96 kHz, 57235892 samples It might be nice if one day file recognised DTS and Dolby Digital encoding within the audio samples of a WAV stream (see [3] for example files), but that would be icing on the cake. Thanks for your work on file! Ben Stanley [1] http://www.2l.musiconline.no/shop/displayAlbum.asp?id=34188 [2] http://www.2l.no/hires/index.html [3] http://www.sr.se/cgi-bin/mall/index.asp?programid=2445 From sds at gnu.org Tue Jun 24 05:32:14 2008 From: sds at gnu.org (Sam Steingold) Date: Mon, 23 Jun 2008 22:32:14 -0400 Subject: clisp fasl magic entry Message-ID: Hi, I am writing in my official capacity as one of the CLISP maintainers. Since 2004-03-27, clisp fasl files start with (|SYSTEM|::|VERSION|\040' instead of (SYSTEM::VERSION\040' found in the file(1) utility distributed by . this is not good - clisp fasl files are not properly identified. While it is conceivable that there are 4 year old clisp-compiled fasl files floating around, I think it is extremely unlikely, therefore I propose that the file(1) maintainers replace the current line 0 string (SYSTEM::VERSION\040' CLISP byte-compiled Lisp program text with 0 string (|SYSTEM|::|VERSION|\040' CLISP byte-compiled Lisp program text for your reference: clisp homepage: http://clisp.cons.org clisp MAGIC.add file: http://clisp.cvs.sourceforge.net/*checkout*/clisp/clisp/unix/MAGIC.add Thanks! -- Sam Steingold (http://sds.podval.org/) on Fedora release 9 (Sulphur) http://palestinefacts.org http://ffii.org http://openvotingconsortium.org http://jihadwatch.org http://israelunderattack.slide.com There are two ways to write error-free programs; only the third one works. From christos at zoulas.com Tue Jun 24 14:58:00 2008 From: christos at zoulas.com (Christos Zoulas) Date: Tue, 24 Jun 2008 07:58:00 -0400 Subject: clisp fasl magic entry In-Reply-To: from Sam Steingold (Jun 23, 10:32pm) Message-ID: <20080624115800.DF2B75654E@rebar.astron.com> On Jun 23, 10:32pm, sds at gnu.org (Sam Steingold) wrote: -- Subject: clisp fasl magic entry | Hi, | | I am writing in my official capacity as one of the CLISP maintainers. | Since 2004-03-27, clisp fasl files start with | (|SYSTEM|::|VERSION|\040' | instead of | (SYSTEM::VERSION\040' | found in the file(1) utility distributed by | . | this is not good - clisp fasl files are not properly identified. | While it is conceivable that there are 4 year old clisp-compiled fasl | files floating around, I think it is extremely unlikely, therefore I | propose that the file(1) maintainers replace the current line | | 0 string (SYSTEM::VERSION\040' CLISP byte-compiled Lisp program text | with | | 0 string (|SYSTEM|::|VERSION|\040' CLISP byte-compiled Lisp program text | | for your reference: | clisp homepage: http://clisp.cons.org | clisp MAGIC.add file: | http://clisp.cvs.sourceforge.net/*checkout*/clisp/clisp/unix/MAGIC.add | | Thanks! | Thanks, I kept both. christos From sds at podval.org Fri Jun 27 17:31:04 2008 From: sds at podval.org (Sam Steingold) Date: Fri, 27 Jun 2008 10:31:04 -0400 Subject: clisp fasl magic entry In-Reply-To: <20080624115800.DF2B75654E@rebar.astron.com> References: <20080624115800.DF2B75654E@rebar.astron.com> Message-ID: <4864F9A8.1020506@podval.org> Christos Zoulas wrote: > On Jun 23, 10:32pm, sds at gnu.org (Sam Steingold) wrote: > -- Subject: clisp fasl magic entry > > | I am writing in my official capacity as one of the CLISP maintainers. > | Since 2004-03-27, clisp fasl files start with > | (|SYSTEM|::|VERSION|\040' > | instead of > | (SYSTEM::VERSION\040' > | found in the file(1) utility distributed by > | . > | this is not good - clisp fasl files are not properly identified. > | While it is conceivable that there are 4 year old clisp-compiled fasl > | files floating around, I think it is extremely unlikely, therefore I > | propose that the file(1) maintainers replace the current line > | > | 0 string (SYSTEM::VERSION\040' CLISP byte-compiled Lisp program text > | with > | > | 0 string (|SYSTEM|::|VERSION|\040' CLISP byte-compiled Lisp program text > | > | for your reference: > | clisp homepage: http://clisp.cons.org > | clisp MAGIC.add file: > | http://clisp.cvs.sourceforge.net/*checkout*/clisp/clisp/unix/MAGIC.add > Thanks, I kept both. Thanks. I hope you mark the old version with a message "CLISP byte-compiled Lisp program text (pre 2004-03-27)" or something like that From christos at zoulas.com Sat Jun 28 17:46:50 2008 From: christos at zoulas.com (Christos Zoulas) Date: Sat, 28 Jun 2008 10:46:50 -0400 Subject: clisp fasl magic entry In-Reply-To: <4864F9A8.1020506@podval.org> from Sam Steingold (Jun 27, 10:31am) Message-ID: <20080628144651.07BB25654E@rebar.astron.com> On Jun 27, 10:31am, sds at podval.org (Sam Steingold) wrote: -- Subject: Re: clisp fasl magic entry | Christos Zoulas wrote: | > On Jun 23, 10:32pm, sds at gnu.org (Sam Steingold) wrote: | > -- Subject: clisp fasl magic entry | > | > | I am writing in my official capacity as one of the CLISP maintainers. | > | Since 2004-03-27, clisp fasl files start with | > | (|SYSTEM|::|VERSION|\040' | > | instead of | > | (SYSTEM::VERSION\040' | > | found in the file(1) utility distributed by | > | . | > | this is not good - clisp fasl files are not properly identified. | > | While it is conceivable that there are 4 year old clisp-compiled fasl | > | files floating around, I think it is extremely unlikely, therefore I | > | propose that the file(1) maintainers replace the current line | > | | > | 0 string (SYSTEM::VERSION\040' CLISP byte-compiled Lisp program text | > | with | > | | > | 0 string (|SYSTEM|::|VERSION|\040' CLISP byte-compiled Lisp program text | > | | > | for your reference: | > | clisp homepage: http://clisp.cons.org | > | clisp MAGIC.add file: | > | http://clisp.cvs.sourceforge.net/*checkout*/clisp/clisp/unix/MAGIC.add | | > Thanks, I kept both. | | Thanks. | I hope you mark the old version with a message | "CLISP byte-compiled Lisp program text (pre 2004-03-27)" | or something like that Will do. christos From christos at zoulas.com Wed Jul 2 19:48:36 2008 From: christos at zoulas.com (Christos Zoulas) Date: Wed, 2 Jul 2008 12:48:36 -0400 Subject: file-4.25 is now available Message-ID: <20080702164837.121FC5654E@rebar.astron.com> As usual in: ftp://ftp.astron.com/pub/file/file-4.25.tar.gz This version features a new experimental !:strength operator. It is currently used in 2 places: - in X3D entries is +1 to beat regular xml - in elf *2 to beat iso9660 Other changes: 2008-07-02 11:15 Christos Zoulas * add !:strength op to adjust magic strength (experimental) 2008-06-16 21:41 Reuben Thomas * Fix automake error in configure.ac. * Add MIME type for Psion Sketch files. 2008-06-05 08:59 Christos Zoulas * Don't print warnings about bad namesize in stripped binaries with PT_NOTE is still there, and the actual note is gone (Jakub Jelinek) 2008-05-28 15:12 Robert Byrnes * magic/Magdir/elf: Note invalid byte order for little-endian SPARC32PLUS. Add SPARC V9 vendor extensions and memory model. * src/elfclass.h: Pass target machine to doshn (for Solaris hardware capabilities). * src/readelf.c (doshn): Add support for Solaris hardware/software capabilities. * src/readelf.h: Ditto. * src/vasprintf.c (dispatch): Add support for ll modifier. 2008-05-16 10:25 Christos Zoulas * Fix compiler warnings. * remove stray printf, and fix a vprintf bug. (Martin Dorey) 2008-05-06 00:13 Robert Byrnes * src/Makefile.am: Ensure that getopt_long and [v]asprintf are included in libmagic, as needed. Remove unnecessary EXTRA_DIST. * src/Makefile.in: Rerun automake. * src/vasprintf.c (dispatch): Fix variable precision bug: be sure to step past '*'. * src/vasprintf.c (core): Remove unreachable code. * src/apprentice.c (set_test_type): Add cast to avoid compiler warning. 2008-04-22 23:45 Christos Zoulas * Add magic submission guidelines (Abel Cheung) * split msdos and windows magic (Abel Cheung) 2008-04-04 11:00 Christos Zoulas * >= <= is not supported, so fix the magic and warn about it. reported by: Thien-Thi Nguyen From david.nachum at berkeley.edu Fri Jul 18 19:26:10 2008 From: david.nachum at berkeley.edu (David Nachum) Date: Fri, 18 Jul 2008 09:26:10 -0700 (PDT) Subject: Changing files in Magdir Message-ID: <1047.209.131.62.113.1216398370.squirrel@calmail.berkeley.edu> Are the files in Magdir read at compile time or run time? How can I ensure/check that changes made take effect when using magic? Thanks! David From christos at zoulas.com Fri Jul 18 20:25:35 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 18 Jul 2008 13:25:35 -0400 Subject: Changing files in Magdir In-Reply-To: <1047.209.131.62.113.1216398370.squirrel@calmail.berkeley.edu> from "David Nachum" (Jul 18, 9:26am) Message-ID: <20080718172535.8310B5654E@rebar.astron.com> On Jul 18, 9:26am, david.nachum at berkeley.edu ("David Nachum") wrote: -- Subject: Changing files in Magdir | Are the files in Magdir read at compile time or run time? How can I | ensure/check that changes made take effect when using magic? | | Thanks! | David Depends on -m arg and $MAGIC. By default, they get compiled and not consulted anymore. christos From david.nachum at berkeley.edu Fri Jul 18 21:30:25 2008 From: david.nachum at berkeley.edu (David Nachum) Date: Fri, 18 Jul 2008 11:30:25 -0700 (PDT) Subject: Changing files in Magdir In-Reply-To: <20080718172535.8310B5654E@rebar.astron.com> References: <20080718172535.8310B5654E@rebar.astron.com> Message-ID: <42241.209.131.62.113.1216405825.squirrel@calmail.berkeley.edu> Thank you for your help. I am still a bit confused though. If I delete sgml (one of the files in Magdir), I can't compile 'file'. However, if I just empty the file, I can compile and run without any problems,and the types that are handled by sgml are still handled correctly. Any idea what's going on? David > On Jul 18, 9:26am, david.nachum at berkeley.edu ("David Nachum") wrote: > -- Subject: Changing files in Magdir > > | Are the files in Magdir read at compile time or run time? How can I > | ensure/check that changes made take effect when using magic? > | > | Thanks! > | David > > Depends on -m arg and $MAGIC. By default, they get compiled and not > consulted > anymore. > > christos > > _______________________________________________ > File mailing list > File at mx.gw.com > http://mx.gw.com/mailman/listinfo/file > From christos at zoulas.com Fri Jul 18 21:33:50 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 18 Jul 2008 14:33:50 -0400 Subject: Changing files in Magdir In-Reply-To: <42241.209.131.62.113.1216405825.squirrel@calmail.berkeley.edu> from "David Nachum" (Jul 18, 11:30am) Message-ID: <20080718183351.030855654E@rebar.astron.com> On Jul 18, 11:30am, david.nachum at berkeley.edu ("David Nachum") wrote: -- Subject: Re: Changing files in Magdir | Thank you for your help. | | I am still a bit confused though. If I delete sgml (one of the files in | Magdir), I can't compile 'file'. However, if I just empty the file, I can | compile and run without any problems,and the types that are handled by | sgml are still handled correctly. | | Any idea what's going on? | | David Well, you have to make clean and make again. If you delete a file in older versions of file, that would break the compilation because all the magic files were listed in the Makefile. This is not the case anymore. christos From wl at gnu.org Fri Jul 25 09:03:35 2008 From: wl at gnu.org (Werner LEMBERG) Date: Fri, 25 Jul 2008 08:03:35 +0200 (CEST) Subject: PNG image not recognized Message-ID: <20080725.080335.37181566.wl@gnu.org> [file 4.25] The attached image isn't properly recognized as PNG. > file micross-01245_winxp.png micross-01245_winxp.png: data Werner -------------- next part -------------- A non-text attachment was scrubbed... Name: micross-01245_winxp.png Type: image/png Size: 2881 bytes Desc: not available URL: From david.nachum at berkeley.edu Mon Jul 28 18:42:03 2008 From: david.nachum at berkeley.edu (David Nachum) Date: Mon, 28 Jul 2008 08:42:03 -0700 (PDT) Subject: Is libmagic multithreaded? Message-ID: <34160.67.161.13.206.1217259723.squirrel@calmail.berkeley.edu> Is libmagic multi-threaded? -David From christos at zoulas.com Mon Jul 28 20:05:18 2008 From: christos at zoulas.com (Christos Zoulas) Date: Mon, 28 Jul 2008 13:05:18 -0400 Subject: Is libmagic multithreaded? In-Reply-To: <34160.67.161.13.206.1217259723.squirrel@calmail.berkeley.edu> from "David Nachum" (Jul 28, 8:42am) Message-ID: <20080728170518.701C15654E@rebar.astron.com> On Jul 28, 8:42am, david.nachum at berkeley.edu ("David Nachum") wrote: -- Subject: Is libmagic multithreaded? | Is libmagic multi-threaded? | -David magic_t contains state, so you cannot use it from multiple threads. But if each thread has it's own magic_t obtained by magic_open, then you are fine. christos From david.nachum at berkeley.edu Fri Aug 15 02:11:55 2008 From: david.nachum at berkeley.edu (David Nachum) Date: Thu, 14 Aug 2008 16:11:55 -0700 (PDT) Subject: magic_load: using default + extra magic files? Message-ID: <18939.209.131.62.113.1218755515.squirrel@calmail.berkeley.edu> At the moment, when using magic_load (libmagic(3)), if the file name used as the second parameter is something other than NULL, then ONLY that file is used (and not the default database file). This means that in order to use one's own magic files in ADDITION to the default, one must retrieve the location of the default magic file (as far as I can tell, the easiest way to do this is the command "file -v") and add that to the colon-separated list of files. Is there an easier way to do this using the libmagic library? Thanks. -David From david.nachum at berkeley.edu Fri Aug 15 17:43:56 2008 From: david.nachum at berkeley.edu (David Nachum) Date: Fri, 15 Aug 2008 07:43:56 -0700 (PDT) Subject: magic_file and magic_buffer yielding different results Message-ID: <36990.209.131.62.115.1218811436.squirrel@calmail.berkeley.edu> magic_file always yields the correct file type, but magic_buffer sometimes doesn't. Also, magic_buffer yields different results based on the buffer "length" you provide it. My intuition is that providing the exact size of the file always should work... but what I'm seeing is that often you get the correct result only when the length is very short (eg 5 bytes). Thanks for the help. David From filemaillist at adaptivetime.com Sun Aug 17 20:57:48 2008 From: filemaillist at adaptivetime.com (Gravis) Date: Sun, 17 Aug 2008 13:57:48 -0400 Subject: file/type list/libmagic overhaul Message-ID: <1218995868.20420.0.camel@base> in short: im proposing a massive update to the file type file as well as change it's format to something faster. my story: in my programming endeavor im dealing with files with no names so naturally i turned to using a library to do magic numbers/patterns to identify the types. however, my instinct to use something in the LSB was sorely met by a rustic and out of date list of file types when i found it cant even detect PNG images. my alternatives were to use a desktop specific lib that had a MUCH more complete list of types or write my own, neither of which are appealing. however, i did notice that both desktop specific and libmagic are SLOW specifically because of how they are organized. my proposal: 1. an online form in which people can submit types to add which would require occasional updates to the list of course (every 3 months?). 2. change the file type list to a binary format (like some of the alternatives to libmagic) 3. make the file type list more like a specialized database (sorted by popularity) for SIGNIFICANTLY faster type lookup times. 4. try to get desktop environments to use libmagic (which they havent due to it's inadequacy) other implimentations: this is how ROX, Gnome and soon(already?) KDE does stores file types: http://www.termalkristaly.hu/doc/shared-mime-info/shared-mime-info-spec.html/x34.html#AEN214 more info at: http://www.termalkristaly.hu/doc/shared-mime-info/shared-mime-info-spec.html because this is in the LSB i think that it should include every file type under the sun including commercial products, game files and even ROM formats for emulators. if this is just too much, an official (RFC documented formats) list and an unofficial DB could be used though i really hope you would consider having all formats in one. i'm willing to do all the programming myself but i really want to get it into the LSB so that once and for all the file type lookup can be unified for all applications. thanks for your time, Gravis From christos at zoulas.com Tue Aug 19 12:01:19 2008 From: christos at zoulas.com (Christos Zoulas) Date: Tue, 19 Aug 2008 05:01:19 -0400 Subject: file/type list/libmagic overhaul In-Reply-To: <1218995868.20420.0.camel@base> from Gravis (Aug 17, 1:57pm) Message-ID: <20080819090119.CE6015654E@rebar.astron.com> On Aug 17, 1:57pm, filemaillist at adaptivetime.com (Gravis) wrote: -- Subject: file/type list/libmagic overhaul | in short: | im proposing a massive update to the file type file as well as change | it's format to something faster. | | my story: | in my programming endeavor im dealing with files with no names so | naturally i turned to using a library to do magic numbers/patterns to | identify the types. however, my instinct to use something in the LSB | was sorely met by a rustic and out of date list of file types when i | found it cant even detect PNG images. my alternatives were to use a | desktop specific lib that had a MUCH more complete list of types or | write my own, neither of which are appealing. however, i did notice | that both desktop specific and libmagic are SLOW specifically because of | how they are organized. | | my proposal: | 1. an online form in which people can submit types to add which would | require occasional updates to the list of course (every 3 months?). That is a good idea, although you'll need a lot of manual fixes because people usually give weak magic descriptions that match ~everything. I would also suggest that people submit sample files so that we can write unit-tests. | 2. change the file type list to a binary format (like some of the | alternatives to libmagic) libmagic uses binary files. The magic.mgc file is pre-parsed during compilation time and it is mmapped at runtime by libmagic. | 3. make the file type list more like a specialized database (sorted by | popularity) for SIGNIFICANTLY faster type lookup times. This does not work because the popular magic may match a more specialized entry pre-maturely and give you the wrong results. Magic is sorted by strength. Finally lots of the performance loss is caused by magic regex's whitespace and string case insensitivity tests, not by load time. | 4. try to get desktop environments to use libmagic (which they havent | due to it's inadequacy) Many of them do. | other implimentations: | this is how ROX, Gnome and soon(already?) KDE does stores file types: | http://www.termalkristaly.hu/doc/shared-mime-info/shared-mime-info-spec.html/x34.html#AEN214 | more info at: | http://www.termalkristaly.hu/doc/shared-mime-info/shared-mime-info-spec.html | | because this is in the LSB i think that it should include every file | type under the sun including commercial products, game files and even | ROM formats for emulators. if this is just too much, an official (RFC | documented formats) list and an unofficial DB could be used though i | really hope you would consider having all formats in one. This is a subset of the functionality that the magic format provides now. For example, this does not provide indirect or relative offsets. | i'm willing to do all the programming myself but i really want to get it | into the LSB so that once and for all the file type lookup can be | unified for all applications. Well, file and the magic format specification has a POSIX definition. Most commercial and non commercial OS's use this implementation of file and I doubt it that they would appreciate a change in the magic format. I appreciate that you want to work on improving file, but please do some more research and come up with concrete ways of improving it without breaking compatibility. christos From filemaillist at adaptivetime.com Tue Aug 19 16:04:59 2008 From: filemaillist at adaptivetime.com (Gravis) Date: Tue, 19 Aug 2008 09:04:59 -0400 Subject: file/type list/libmagic overhaul In-Reply-To: <20080819090119.CE6015654E@rebar.astron.com> References: <20080819090119.CE6015654E@rebar.astron.com> Message-ID: <1219151099.30038.28.camel@base> On Tue, 2008-08-19 at 05:01 -0400, Christos Zoulas wrote: > On Aug 17, 1:57pm, filemaillist at adaptivetime.com (Gravis) wrote: > -- Subject: file/type list/libmagic overhaul > | > | my proposal: > | 1. an online form in which people can submit types to add which would > | require occasional updates to the list of course (every 3 months?). > > That is a good idea, although you'll need a lot of manual fixes because > people usually give weak magic descriptions that match ~everything. I > would also suggest that people submit sample files so that we can write > unit-tests. i agree that people will give weak descriptions which is why i think an online "bombardment test" script would be a good idea. basically, a repository of known format files that the submitted magic description would be tested against. of course they will need to upload a file they are looking to identify. assuming they get passed a basic test that it doesnt misidentify files, it would be tested on a larger more extensive collection of files. any way it goes, there needs to be a way of getting formats added to the list. > Well, file and the magic format specification has a POSIX definition. Most > commercial and non commercial OS's use this implementation of file and I > doubt it that they would appreciate a change in the magic format. hmm... i didnt realize it was posix. do you have a copy of the spec i could have? from what i can tell the only way to get a copy from IEEE is to be a member of IEEE, which i am not. if speed truly isnt an issue here, i guess changing the file format is a moot point. anyway, my main concern is to get more formats added because not having basic file magic descriptions like one for PNG is just ridiculous. -- Gravis From ian at darwinsys.com Tue Aug 19 15:36:04 2008 From: ian at darwinsys.com (Ian Darwin) Date: Tue, 19 Aug 2008 08:36:04 -0400 Subject: file/type list/libmagic overhaul In-Reply-To: <20080819090119.CE6015654E@rebar.astron.com> References: <20080819090119.CE6015654E@rebar.astron.com> Message-ID: <48AABE34.9020300@darwinsys.com> > That is a good idea, although you'll need a lot of manual fixes because > people usually give weak magic descriptions that match ~everything. I > would also suggest that people submit sample files so that we can write > unit-tests. Start with the regress tests from OpenBSD's repository - src/regress/usr.bin/file/*. The Makefile looks like this: # $OpenBSD: Makefile,v 1.1 2006/06/02 01:07:44 ian Exp $ FILE=file # temporarily excluded - t23 REGRESS_TARGETS=t0 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t15 t17 t18 t20 t21 t22 t30 # .in: input file # .out: desired result # t0 empty # t1 archive # t2 ascii archive # t3 ranlib # t4 awk # t5 awk # t6 csh # t7 awk # t8 interpreter # t9 java class # t10 java serialized data # t11 mach-o binary # t15 tar archive # t17 C program # t20 HTML # t21 SGML # t22 XML # t23 raw xml all: ${REGRESS_TARGET} .SUFFIXES: .in .in: @echo ${*} @${FILE} ${.CURDIR}/${*}.in | \ sed -e "s@${.CURDIR}/@@" | \ diff - ${.CURDIR}/${*}.out || \ (echo "XXX ${*} failed" && false) clean: .include From wl at gnu.org Tue Aug 19 16:43:45 2008 From: wl at gnu.org (Werner LEMBERG) Date: Tue, 19 Aug 2008 15:43:45 +0200 (CEST) Subject: file/type list/libmagic overhaul In-Reply-To: <1219151099.30038.28.camel@base> References: <20080819090119.CE6015654E@rebar.astron.com> <1219151099.30038.28.camel@base> Message-ID: <20080819.154345.213290650.wl@gnu.org> > anyway, my main concern is to get more formats added because not > having basic file magic descriptions like one for PNG is just > ridiculous. Not recognizing PNG is a bug of the latest `file' version; former versions worked fine. I've reported this already, but got no response. Werner From christos at zoulas.com Wed Aug 20 11:02:53 2008 From: christos at zoulas.com (Christos Zoulas) Date: Wed, 20 Aug 2008 04:02:53 -0400 Subject: file/type list/libmagic overhaul In-Reply-To: <1219151099.30038.28.camel@base> from Gravis (Aug 19, 9:04am) Message-ID: <20080820080253.C94665654E@rebar.astron.com> On Aug 19, 9:04am, filemaillist at adaptivetime.com (Gravis) wrote: -- Subject: Re: file/type list/libmagic overhaul | > That is a good idea, although you'll need a lot of manual fixes because | > people usually give weak magic descriptions that match ~everything. I | > would also suggest that people submit sample files so that we can write | > unit-tests. | | i agree that people will give weak descriptions which is why i think an | online "bombardment test" script would be a good idea. basically, a | repository of known format files that the submitted magic description | would be tested against. of course they will need to upload a file they | are looking to identify. assuming they get passed a basic test that it | doesnt misidentify files, it would be tested on a larger more extensive | collection of files. any way it goes, there needs to be a way of | getting formats added to the list. Right. This is why we need to start collecting samples, so that we can perform such tests. | > Well, file and the magic format specification has a POSIX definition. Most | > commercial and non commercial OS's use this implementation of file and I | > doubt it that they would appreciate a change in the magic format. | | hmm... i didnt realize it was posix. do you have a copy of the spec i | could have? from what i can tell the only way to get a copy from IEEE | is to be a member of IEEE, which i am not. if speed truly isnt an issue | here, i guess changing the file format is a moot point. most posix stuff is free now (except the compliance tests): http://www.opengroup.org/onlinepubs/009695399/utilities/file.html | anyway, my main concern is to get more formats added because not having | basic file magic descriptions like one for PNG is just ridiculous. I don't know why you say this; here's the current entry for PNG from images: # PNG [Portable Network Graphics, or "PNG's Not GIF"] images # (Greg Roelofs, newt at uchicago.edu) # (Albert Cahalan, acahalan at cs.uml.edu) # # 137 P N G \r \n ^Z \n [4-byte length] H E A D [HEAD data] [HEAD crc] ... # 0 string \x89PNG\x0d\x0a\x1a\x0a PNG image !:mime image/png >>16 belong x \b, %ld x >>20 belong x %ld, >>24 byte x %d-bit >>25 byte 0 grayscale, >>25 byte 2 \b/color RGB, >>25 byte 3 colormap, >>25 byte 4 gray+alpha, >>25 byte 6 \b/color RGBA, #>>26 byte 0 deflate/32K, >>28 byte 0 non-interlaced >>28 byte 1 interlaced christos From wl at gnu.org Wed Aug 20 15:41:27 2008 From: wl at gnu.org (Werner LEMBERG) Date: Wed, 20 Aug 2008 14:41:27 +0200 (CEST) Subject: file/type list/libmagic overhaul In-Reply-To: <20080820080253.C94665654E@rebar.astron.com> References: <1219151099.30038.28.camel@base> <20080820080253.C94665654E@rebar.astron.com> Message-ID: <20080820.144127.73262658.wl@gnu.org> > | anyway, my main concern is to get more formats added because not > | having basic file magic descriptions like one for PNG is just > | ridiculous. > > I don't know why you say this; here's the current entry for PNG from > images: I've reported this already to the list, but my mail has either went unnoticed or Christos hasn't received it. BTW, Christos, it's not possible to send a mail directly to ; I get this: host relay.astron.com[208.77.212.97]: 554 : Client host rejected: Access denied due to mail system abuse Werner From wl at gnu.org Wed Aug 20 15:50:43 2008 From: wl at gnu.org (Werner LEMBERG) Date: Wed, 20 Aug 2008 14:50:43 +0200 (CEST) Subject: file/type list/libmagic overhaul In-Reply-To: <20080820.144127.73262658.wl@gnu.org> References: <1219151099.30038.28.camel@base> <20080820080253.C94665654E@rebar.astron.com> <20080820.144127.73262658.wl@gnu.org> Message-ID: <20080820.145043.164039348.wl@gnu.org> > I've reported this already to the list, but my mail has either went > unnoticed .... uh, oh, I shouldn't make phone calls and write emails at same time :-) I meant: .. has either gone unnoticed... Werner From christos at zoulas.com Wed Aug 20 17:11:31 2008 From: christos at zoulas.com (Christos Zoulas) Date: Wed, 20 Aug 2008 10:11:31 -0400 Subject: file/type list/libmagic overhaul In-Reply-To: <20080820.144127.73262658.wl@gnu.org> from Werner LEMBERG (Aug 20, 2:41pm) Message-ID: <20080820141131.37C075654E@rebar.astron.com> On Aug 20, 2:41pm, wl at gnu.org (Werner LEMBERG) wrote: -- Subject: Re: file/type list/libmagic overhaul | > | anyway, my main concern is to get more formats added because not | > | having basic file magic descriptions like one for PNG is just | > | ridiculous. | > | > I don't know why you say this; here's the current entry for PNG from | > images: | | I've reported this already to the list, but my mail has either went | unnoticed or Christos hasn't received it. | | BTW, Christos, it's not possible to send a mail directly to | ; I get this: | | host relay.astron.com[208.77.212.97]: | 554 : Client host rejected: | Access denied due to mail system abuse | I looked for your server in spamhaus and dsbl and I don't see it listed. I cannot explain it. christos From filemaillist at adaptivetime.com Wed Aug 20 20:15:01 2008 From: filemaillist at adaptivetime.com (Gravis) Date: Wed, 20 Aug 2008 13:15:01 -0400 Subject: file/type list/libmagic overhaul In-Reply-To: <20080820080253.C94665654E@rebar.astron.com> References: <20080820080253.C94665654E@rebar.astron.com> Message-ID: <1219252501.2968.47.camel@base> On Wed, 2008-08-20 at 04:02 -0400, Christos Zoulas wrote: > On Aug 19, 9:04am, filemaillist at adaptivetime.com (Gravis) wrote: > -- Subject: Re: file/type list/libmagic overhaul > > | i agree that people will give weak descriptions which is why i think an > | online "bombardment test" script would be a good idea. basically, a > | repository of known format files that the submitted magic description > | would be tested against. of course they will need to upload a file they > | are looking to identify. assuming they get passed a basic test that it > | doesnt misidentify files, it would be tested on a larger more extensive > | collection of files. any way it goes, there needs to be a way of > | getting formats added to the list. > > Right. This is why we need to start collecting samples, so that we can > perform such tests. great! > | hmm... i didnt realize it was posix. do you have a copy of the spec i > | could have? from what i can tell the only way to get a copy from IEEE > | is to be a member of IEEE, which i am not. if speed truly isnt an issue > | here, i guess changing the file format is a moot point. > > most posix stuff is free now (except the compliance tests): thanks for the link :) > | anyway, my main concern is to get more formats added because not having > | basic file magic descriptions like one for PNG is just ridiculous. > > I don't know why you say this; here's the current entry for PNG from > images: i did some investigating using your provided sample and found something interesting. whatever the default magic file is, it's not the correct magic file. image used: http://www.libpng.org/pub/png/img_png/pngnow.png ====== console ====== ==== default file ==== gravis at base:~$ file ~/Desktop/pngnow.png /home/gravis/Desktop/pngnow.png: data gravis at base:~$ file --mime-type ~/Desktop/pngnow.png /home/gravis/Desktop/pngnow.png: application/octet-stream gravis at base:~$ file --mime-encoding ~/Desktop/pngnow.png Segmentation fault ==== specified file ==== gravis at base:~$ file -m /usr/share/file/magic.mime ~/Desktop/pngnow.png /home/gravis/Desktop/pngnow.png: image/png gravis at base:~$ file -m /usr/share/file/magic.mime --mime-type ~/Desktop/pngnow.png /home/gravis/Desktop/pngnow.png: application/octet-stream gravis at base:~$ file -m /usr/share/file/magic.mime --mime-encoding ~/Desktop/pngnow.png /home/gravis/Desktop/pngnow.png: ====== end console ====== it seems like this may be a distribution issue. i use debian so i guess i'll talk to the package maintainer and perhaps check some other distributions for the same foul up. any comments? hey Werner, what distribution are you using? i'll talk to the gnome/kde people and find out their gripes. perhaps an exhaustive list of formats may sway them into using libmagic. im feeling much better about all of this now. -- Gravis From wl at gnu.org Wed Aug 20 20:28:51 2008 From: wl at gnu.org (Werner LEMBERG) Date: Wed, 20 Aug 2008 19:28:51 +0200 (CEST) Subject: file/type list/libmagic overhaul In-Reply-To: <1219252501.2968.47.camel@base> References: <20080820080253.C94665654E@rebar.astron.com> <1219252501.2968.47.camel@base> Message-ID: <20080820.192851.122319444.wl@gnu.org> > it seems like this may be a distribution issue. Hmm. > hey Werner, what distribution are you using? I compiled directly from the source. Werner From wl at gnu.org Wed Aug 20 22:12:21 2008 From: wl at gnu.org (Werner LEMBERG) Date: Wed, 20 Aug 2008 21:12:21 +0200 (CEST) Subject: file/type list/libmagic overhaul In-Reply-To: <20080820141131.37C075654E@rebar.astron.com> References: <20080820.144127.73262658.wl@gnu.org> <20080820141131.37C075654E@rebar.astron.com> Message-ID: <20080820.211221.144436995.wl@gnu.org> > I looked for your server in spamhaus and dsbl and I don't see it listed. I > cannot explain it. Which one? gnu.org? Or onlinehome.de? Werner From christos at zoulas.com Fri Aug 22 18:30:46 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 22 Aug 2008 11:30:46 -0400 Subject: file/type list/libmagic overhaul In-Reply-To: <20080820.211221.144436995.wl@gnu.org> from Werner LEMBERG (Aug 20, 9:12pm) Message-ID: <20080822153046.885D45654F@rebar.astron.com> On Aug 20, 9:12pm, wl at gnu.org (Werner LEMBERG) wrote: -- Subject: Re: file/type list/libmagic overhaul | | > I looked for your server in spamhaus and dsbl and I don't see it listed. I | > cannot explain it. | | Which one? gnu.org? Or onlinehome.de? I don't remember right now. The mail server from the bounce... christos From cloos+file at jhcloos.com Sun Aug 24 02:15:27 2008 From: cloos+file at jhcloos.com (James Cloos) Date: Sat, 23 Aug 2008 19:15:27 -0400 Subject: file/type list/libmagic overhaul In-Reply-To: <1219252501.2968.47.camel@base> (Gravis's message of "Wed, 20 Aug 2008 13:15:01 -0400") References: <20080820080253.C94665654E@rebar.astron.com> <1219252501.2968.47.camel@base> Message-ID: >>>>> "Gravis" == Gravis writes: Gravis> i did some investigating using your provided sample and found Gravis> something interesting. whatever the default magic file is, it's Gravis> not the correct magic file. I just tried my installed instance of file(1) (gentoo box, compiled via the ebuild using gcc 4.2; box is x86) against a text magic(5) file. Specifically, I unpacked the 4.25 tar.gz and pointed file at the images file in Magdir. It still cannot detect a PNG file, though everything else I tested worked as expected: correct detection if in the specified Magdir file or a default (such as ASCII Text for ppm, pdf, ps files) if the file was not included in the Magdir file I specified. The images file says, in part: ,----< excerpt from file-4.25/Magdir/images > | 0 string \x89PNG\x0d\x0a\x1a\x0a PNG image `---- If I drop the \x0a from the end of the string the detection works. This, even though: ,----< dd bs=8 count=1 in=foo.png|od -t x1 > | 0000000 89 50 4e 47 0d 0a 1a 0a `---- So there seems to be a bug in how file (or libmagic?) handles strings. That change in the magic(5) file also prevents the segfault when calling file(1) with the --mime-encoding option. Gdb says about the segv: ,----[ backtrace from ?file --mime-encoding foo.png? ] | #0 0xb7f997a3 in strlen () from /lib/libc.so.6 | #1 0xb80717cf in file_getbuffer () from /usr/lib/libmagic.so.1 | #2 0xb80653ce in file_or_fd () from /usr/lib/libmagic.so.1 | #3 0x08048dbe in process () | #4 0x080496d6 in main () `---- All works flawlessly with a seven octet string in magic(5) for PNG; both bugs only show up with the full eight octet string. -JimC -- James Cloos OpenPGP: 1024D/ED7DAEA6 From cloos+file at jhcloos.com Sun Aug 24 03:24:03 2008 From: cloos+file at jhcloos.com (James Cloos) Date: Sat, 23 Aug 2008 20:24:03 -0400 Subject: file/type list/libmagic overhaul In-Reply-To: (James Cloos's message of "Sat, 23 Aug 2008 19:15:27 -0400") References: <20080820080253.C94665654E@rebar.astron.com> <1219252501.2968.47.camel@base> Message-ID: > All works flawlessly with a seven octet string in magic(5) for PNG; Actually I spoke too soon. I had commented out the >> lines while determining what change would make it work. Uncommenting them, however, did nothing. The only way I could make the >> lines do anything was to change at least the first >> line into a > line. Eg: ,----[ excerpt from unidiff ] | - >>16 belong x \b, %ld | + >16 belong x \b, %ld `---- So it doesn't seem to want to check level++++ w/o first having checked level++. -JimC -- James Cloos OpenPGP: 1024D/ED7DAEA6 From ordo.ad at gmail.com Wed Aug 27 12:49:13 2008 From: ordo.ad at gmail.com (Alessandro Doro) Date: Wed, 27 Aug 2008 11:49:13 +0200 Subject: [christos at zoulas.com: Re: file 4.25 doesn't recognize PNG image files] Message-ID: <20080827094913.GA4972@PIV> Patch attached. I don't know if the changes in softmagic.c affect other parts of the code. A temporary fix with no changes in the code is to substitute line 78 in magic/Magdir/images: 0 string \x89PNG\x0d\x0a\x1a\x0a PNG image with: 0 bequad 0x89504e470d0a1a0a PNG image and also substitute the occurences of '>>' with '>' (lines 80-90). bye, Alessandro diff -ur file-4.25.orig/magic/Magdir/images file-4.25/magic/Magdir/images --- file-4.25.orig/magic/Magdir/images 2008-07-02 17:16:52.000000000 +0200 +++ file-4.25/magic/Magdir/images 2008-08-27 11:17:46.000000000 +0200 @@ -77,17 +77,17 @@ # 0 string \x89PNG\x0d\x0a\x1a\x0a PNG image !:mime image/png ->>16 belong x \b, %ld x ->>20 belong x %ld, ->>24 byte x %d-bit ->>25 byte 0 grayscale, ->>25 byte 2 \b/color RGB, ->>25 byte 3 colormap, ->>25 byte 4 gray+alpha, ->>25 byte 6 \b/color RGBA, -#>>26 byte 0 deflate/32K, ->>28 byte 0 non-interlaced ->>28 byte 1 interlaced +>16 belong x \b, %ld x +>20 belong x %ld, +>24 byte x %d-bit +>25 byte 0 grayscale, +>25 byte 2 \b/color RGB, +>25 byte 3 colormap, +>25 byte 4 gray+alpha, +>25 byte 6 \b/color RGBA, +#>26 byte 0 deflate/32K, +>28 byte 0 non-interlaced +>28 byte 1 interlaced # possible GIF replacements; none yet released! # (Greg Roelofs, newt at uchicago.edu) diff -ur file-4.25.orig/src/softmagic.c file-4.25/src/softmagic.c --- file-4.25.orig/src/softmagic.c 2008-04-29 03:10:49.000000000 +0200 +++ file-4.25/src/softmagic.c 2008-08-27 11:19:33.000000000 +0200 @@ -665,8 +665,8 @@ /* Null terminate and eat *trailing* return */ p->s[sizeof(p->s) - 1] = '\0'; len = strlen(p->s); - if (len-- && p->s[len] == '\n') - p->s[len] = '\0'; + /*if (len-- && p->s[len] == '\n') + p->s[len] = '\0';*/ return 1; } case FILE_PSTRING: { @@ -678,8 +678,8 @@ *ptr1++ = *ptr2++; *ptr1 = '\0'; len = strlen(p->s); - if (len-- && p->s[len] == '\n') - p->s[len] = '\0'; + /*if (len-- && p->s[len] == '\n') + p->s[len] = '\0';*/ return 1; } case FILE_BESHORT: ----- Forwarded message from Christos Zoulas ----- Delivered-To: ordo.ad at gmail.com Authentication-Results: mx.google.com; spf=neutral (google.com: 38.117.134.202 is neither permitted nor denied by best guess record for domain of christos at zoulas.com) smtp.mail=christos at zoulas.com From: Christos Zoulas Date: Fri, 18 Jul 2008 18:05:49 -0400 In-Reply-To: <20080718205242.GA20344 at PIV> from Alessandro Doro (Jul 18, 10:52pm) Organization: Astron Software X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: Alessandro Doro Subject: Re: file 4.25 doesn't recognize PNG image files On Jul 18, 10:52pm, ordo.ad at gmail.com (Alessandro Doro) wrote: -- Subject: Re: file 4.25 doesn't recognize PNG image files | On Fri, Jul 18, 2008 at 03:15:33AM +0200, Alessandro Doro wrote: | > The PNG signature isn't matched (see line 2055 in file.debug-4.25) | > against: | > string \x89PNG\x0d\x0a\x1a\x0a | > | > but is matched against: | > bequad 0x89504e470d0a1a0a | > | > Note the two newline characters in the PNG signature. Is some string | > read or compared in text mode? | > | > | > | > I tried an experiment with this line in the magic file: | > | > string \x89PNG\x0d\x0b\x1a\x0b PNG image | | Well. I found (tested) that the problem is in the last newline character | in: | string \x89PNG\x0d\x0a\x1a\x0a PNG image | | | I also see this code in the function mconvert() in softmagic.c, | lines 660-684: | | case FILE_STRING: | case FILE_BESTRING16: | case FILE_LESTRING16: { | size_t len; | | /* Null terminate and eat *trailing* return */ | p->s[sizeof(p->s) - 1] = '\0'; | len = strlen(p->s); | if (len-- && p->s[len] == '\n') | p->s[len] = '\0'; | return 1; | } | case FILE_PSTRING: { | char *ptr1 = p->s, *ptr2 = ptr1 + 1; | size_t len = *p->s; | if (len >= sizeof(p->s)) | len = sizeof(p->s) - 1; | while (len--) | *ptr1++ = *ptr2++; | *ptr1 = '\0'; | len = strlen(p->s); | if (len-- && p->s[len] == '\n') | p->s[len] = '\0'; | return 1; | } | | I don't know the purpose of eating the trailing return (newline), but | commenting out the lines: | if (len-- && p->s[len] == '\n') | p->s[len] = '\0'; You are absolutely right. I will fix it! | makes the program recognize png images in my ArchLinux system. | Maybe this can be a starting point. | | The same code is in version 4.24 but it doesn't show up because the | match is agaisnt the 4 byte string "\x89PNG". | The following 4 bytes are compared in the level 1 test against the | belong 0x0d0a1a0a. | | | I also have to point out the other bug in the magic file: the lines | after "!:mime image/png" should contain only one '>' sign because there | isn't a level 1 test. | | Without this correction the program prints only: | test.png: PNG image | leaving out the image details. | | | bye Good catch! christos ----- End forwarded message ----- -------------- next part -------------- diff -ur file-4.25.orig/magic/Magdir/images file-4.25/magic/Magdir/images --- file-4.25.orig/magic/Magdir/images 2008-07-02 17:16:52.000000000 +0200 +++ file-4.25/magic/Magdir/images 2008-08-27 11:17:46.000000000 +0200 @@ -77,17 +77,17 @@ # 0 string \x89PNG\x0d\x0a\x1a\x0a PNG image !:mime image/png ->>16 belong x \b, %ld x ->>20 belong x %ld, ->>24 byte x %d-bit ->>25 byte 0 grayscale, ->>25 byte 2 \b/color RGB, ->>25 byte 3 colormap, ->>25 byte 4 gray+alpha, ->>25 byte 6 \b/color RGBA, -#>>26 byte 0 deflate/32K, ->>28 byte 0 non-interlaced ->>28 byte 1 interlaced +>16 belong x \b, %ld x +>20 belong x %ld, +>24 byte x %d-bit +>25 byte 0 grayscale, +>25 byte 2 \b/color RGB, +>25 byte 3 colormap, +>25 byte 4 gray+alpha, +>25 byte 6 \b/color RGBA, +#>26 byte 0 deflate/32K, +>28 byte 0 non-interlaced +>28 byte 1 interlaced # possible GIF replacements; none yet released! # (Greg Roelofs, newt at uchicago.edu) diff -ur file-4.25.orig/src/softmagic.c file-4.25/src/softmagic.c --- file-4.25.orig/src/softmagic.c 2008-04-29 03:10:49.000000000 +0200 +++ file-4.25/src/softmagic.c 2008-08-27 11:19:33.000000000 +0200 @@ -665,8 +665,8 @@ /* Null terminate and eat *trailing* return */ p->s[sizeof(p->s) - 1] = '\0'; len = strlen(p->s); - if (len-- && p->s[len] == '\n') - p->s[len] = '\0'; + /*if (len-- && p->s[len] == '\n') + p->s[len] = '\0';*/ return 1; } case FILE_PSTRING: { @@ -678,8 +678,8 @@ *ptr1++ = *ptr2++; *ptr1 = '\0'; len = strlen(p->s); - if (len-- && p->s[len] == '\n') - p->s[len] = '\0'; + /*if (len-- && p->s[len] == '\n') + p->s[len] = '\0';*/ return 1; } case FILE_BESHORT: From christos at zoulas.com Sat Aug 30 13:10:07 2008 From: christos at zoulas.com (Christos Zoulas) Date: Sat, 30 Aug 2008 06:10:07 -0400 Subject: file-4.26 is now available Message-ID: <20080830101007.648725654E@rebar.astron.com> In the usual place: ftp://ftp.astron.com/pub/file/file-4.26.tar.gz Enjoy, christos 2008-08-30 12:54 Christos Zoulas * Don't eat trailing \n in magic enties. * Cast defines to allow compilation using a c++ compiler. 2008-07-26 00:59 Reuben Thomas * Add MIME types for special files. * Use access to give more accurate information for files that can't be opened. * Add a TODO list. From lists-file at jay.fm Sun Aug 31 16:28:04 2008 From: lists-file at jay.fm (Jay Levitt) Date: Sun, 31 Aug 2008 09:28:04 -0400 Subject: 4.25 and 4.26 won't run on FC9; give spurious "magic version" error Message-ID: <48BA9C64.1030306@jay.fm> Now I'm on a Fedora Core 9 box, x86_64. I can build both 4.25 and 4.26 successfully, but both give the same odd error when run: [root at micro]/usr/local/src/tools/file-4.26/src/.libs# ./file -v file-4.26 magic file from /usr/local/share/file/magic [root at micro]/usr/local/src/tools/file-4.26/src/.libs# ./file ./file file: File 4.23 supports only 4 version magic files. `/usr/local/share/file/magic.mgc' is version 6 Interestingly, the wrapper script DOES work... [root at micro]/usr/local/src/tools/file-4.26/src/.libs# cd .. [root at micro]/usr/local/src/tools/file-4.26/src# ./file -v lt-file-4.26 magic file from /usr/local/share/file/magic [root at micro]/usr/local/src/tools/file-4.26/src# ./file file file: POSIX shell script text executable I'm lost... Jay Levitt From lists-file at jay.fm Sun Aug 31 16:19:23 2008 From: lists-file at jay.fm (Jay Levitt) Date: Sun, 31 Aug 2008 09:19:23 -0400 Subject: Can't build 4.26 on Mac OS X: duplicate symbol _li Message-ID: <48BA9A5B.50007@jay.fm> I downloaded 4.26, but on my Mac (OS X 10.5.4 Leopard), I get a duplicate symbol error from ld. I don't know enough about object modules to debug unaided, but if someone can let me know how to troubleshoot this, I'm happy to dig further. 4.25 builds fine on the same machine; 4.26 builds on my Fedora machine (but see my next post about strange problems running it there). I also tried: make distclean autoreconf ./configure make but the same thing happened. There's an interesting bug at http://issues.sxemacs.org/show_bug.cgi?id=42 which may be related; apparently Apple's gcc does something different with inlines and C99? My C-fu is rusty (and predates C99). The error: ld: duplicate symbol _li in .libs/apprentice.o and .libs/magic.o Toolchain (copyrights snipped): macpro% gcc --version i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5465) macpro% ld -v @(#)PROGRAM:ld PROJECT:ld64-77 macpro% m4 --version GNU M4 1.4.6 macpro% make --version GNU Make 3.81 The full configure/make output: macpro% ./configure checking for a BSD-compatible install... /opt/local/bin/ginstall -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /opt/local/bin/gmkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for builtin ELF support... yes checking for ELF core file support... yes checking for file formats in man section 5... no checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking whether gcc and cc understand -c and -o together... yes checking for a BSD-compatible install... /opt/local/bin/ginstall -c checking whether ln -s works... yes checking build system type... i386-apple-darwin9.4.0 checking host system type... i386-apple-darwin9.4.0 checking for a sed that does not truncate output... /usr/bin/sed checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for ld used by gcc... /usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld checking if the linker (/usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld) is GNU ld... no checking for /usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld option to reload object files... -r checking for BSD-compatible nm... /usr/bin/nm -p checking how to recognise dependent libraries... pass_all checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking dlfcn.h usability... yes checking dlfcn.h presence... yes checking for dlfcn.h... yes checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking dependency style of g++... gcc3 checking how to run the C++ preprocessor... g++ -E checking for g77... no checking for xlf... no checking for f77... no checking for frt... no checking for pgf77... no checking for cf77... no checking for fort77... no checking for fl32... no checking for af77... no checking for xlf90... no checking for f90... no checking for pgf90... no checking for pghpf... no checking for epcf90... no checking for gfortran... no checking for g95... no checking for xlf95... no checking for f95... no checking for fort... no checking for ifort... no checking for ifc... no checking for efc... no checking for pgf95... no checking for lf95... no checking for ftn... no checking whether we are using the GNU Fortran 77 compiler... no checking whether accepts -g... no checking the maximum length of command line arguments... 196608 checking command to parse /usr/bin/nm -p output from gcc object... rm: conftest.dSYM: is a directory rm: conftest.dSYM: is a directory rm: conftest.dSYM: is a directory rm: conftest.dSYM: is a directory ok checking for objdir... .libs checking for ar... ar checking for ranlib... ranlib checking for strip... strip rm: conftest.dSYM: is a directory rm: conftest.dSYM: is a directory checking if gcc static flag works... rm: conftest.dSYM: is a directory yes checking if gcc supports -fno-rtti -fno-exceptions... rm: conftest.dSYM: is a directory no checking for gcc option to produce PIC... -fno-common checking if gcc PIC flag -fno-common works... rm: conftest.dSYM: is a directory yes checking if gcc supports -c -o file.o... rm: conftest.dSYM: is a directory yes checking whether the gcc linker (/usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld) supports shared libraries... yes checking dynamic linker characteristics... darwin9.4.0 dyld checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes configure: creating libtool appending configuration tag "CXX" to libtool rm: conftest.dSYM: is a directory rm: conftest.dSYM: is a directory checking for ld used by g++... /usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld checking if the linker (/usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld) is GNU ld... no checking whether the g++ linker (/usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld) supports shared libraries... yes checking for g++ option to produce PIC... -fno-common checking if g++ PIC flag -fno-common works... rm: conftest.dSYM: is a directory yes checking if g++ supports -c -o file.o... rm: conftest.dSYM: is a directory yes checking whether the g++ linker (/usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld) supports shared libraries... yes checking dynamic linker characteristics... darwin9.4.0 dyld checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes appending configuration tag "F77" to libtool checking for ANSI C header files... (cached) yes checking whether sys/types.h defines makedev... yes checking for sys/wait.h that is POSIX.1 compatible... yes checking for stdint.h... (cached) yes checking fcntl.h usability... yes checking fcntl.h presence... yes checking for fcntl.h... yes checking locale.h usability... yes checking locale.h presence... yes checking for locale.h... yes checking for stdint.h... (cached) yes checking for inttypes.h... (cached) yes checking for unistd.h... (cached) yes checking utime.h usability... yes checking utime.h presence... yes checking for utime.h... yes checking wchar.h usability... yes checking wchar.h presence... yes checking for wchar.h... yes checking wctype.h usability... yes checking wctype.h presence... yes checking for wctype.h... yes checking limits.h usability... yes checking limits.h presence... yes checking for limits.h... yes checking getopt.h usability... yes checking getopt.h presence... yes checking for getopt.h... yes checking err.h usability... yes checking err.h presence... yes checking for err.h... yes checking sys/mman.h usability... yes checking sys/mman.h presence... yes checking for sys/mman.h... yes checking for sys/stat.h... (cached) yes checking for sys/types.h... (cached) yes checking sys/utime.h usability... no checking sys/utime.h presence... no checking for sys/utime.h... no checking sys/time.h usability... yes checking sys/time.h presence... yes checking for sys/time.h... yes checking zlib.h usability... yes checking zlib.h presence... yes checking for zlib.h... yes checking for an ANSI C-conforming const... yes checking for off_t... yes checking for size_t... yes checking for struct stat.st_rdev... yes checking whether struct tm is in sys/time.h or time.h... time.h checking for tm_zone in struct tm... yes checking for tzname... yes checking for tm_isdst in struct tm... yes checking for daylight... yes checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... no rm: conftest.dSYM: is a directory checking for _LARGEFILE_SOURCE value needed for large files... no rm: conftest.dSYM: is a directory checking for mbstate_t... yes checking for uint8_t... yes checking for uint16_t... yes checking for uint32_t... yes checking for int32_t... yes checking for uint64_t... yes checking for int64_t... yes checking for long long... yes checking size of long long... 8 checking for gcc compiler warnings... yes checking for mmap... yes checking for strerror... yes checking for strndup... no checking for strtoul... yes checking for mbrtowc... yes checking for mkstemp... yes checking for utimes... yes checking for utime... yes checking for wcwidth... yes checking for strtof... yes checking for getopt_long... yes checking for asprintf... yes checking for vasprintf... yes checking for gzopen in -lz... yes configure: creating ./config.status config.status: creating Makefile config.status: creating src/Makefile config.status: creating magic/Makefile config.status: creating tests/Makefile config.status: creating doc/Makefile config.status: creating python/Makefile config.status: creating config.h config.status: config.h is unchanged config.status: executing depfiles commands macpro% make make all-recursive Making all in src /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT magic.lo -MD -MP -MF .deps/magic.Tpo -c -o magic.lo magic.c mkdir .libs gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT magic.lo -MD -MP -MF .deps/magic.Tpo -c magic.c -fno-common -DPIC -o .libs/magic.o gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT magic.lo -MD -MP -MF .deps/magic.Tpo -c magic.c -o magic.o >/dev/null 2>&1 mv -f .deps/magic.Tpo .deps/magic.Plo /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT apprentice.lo -MD -MP -MF .deps/apprentice.Tpo -c -o apprentice.lo apprentice.c gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT apprentice.lo -MD -MP -MF .deps/apprentice.Tpo -c apprentice.c -fno-common -DPIC -o .libs/apprentice.o apprentice.c: In function 'load_1': apprentice.c:1699: warning: 'slen' may be used uninitialized in this function gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT apprentice.lo -MD -MP -MF .deps/apprentice.Tpo -c apprentice.c -o apprentice.o >/dev/null 2>&1 mv -f .deps/apprentice.Tpo .deps/apprentice.Plo /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT softmagic.lo -MD -MP -MF .deps/softmagic.Tpo -c -o softmagic.lo softmagic.c gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT softmagic.lo -MD -MP -MF .deps/softmagic.Tpo -c softmagic.c -fno-common -DPIC -o .libs/softmagic.o softmagic.c: In function 'mcopy': softmagic.c:848: warning: format '%zu' expects type 'size_t', but argument 3 has type 'uint32_t' gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT softmagic.lo -MD -MP -MF .deps/softmagic.Tpo -c softmagic.c -o softmagic.o >/dev/null 2>&1 mv -f .deps/softmagic.Tpo .deps/softmagic.Plo /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT ascmagic.lo -MD -MP -MF .deps/ascmagic.Tpo -c -o ascmagic.lo ascmagic.c gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT ascmagic.lo -MD -MP -MF .deps/ascmagic.Tpo -c ascmagic.c -fno-common -DPIC -o .libs/ascmagic.o gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT ascmagic.lo -MD -MP -MF .deps/ascmagic.Tpo -c ascmagic.c -o ascmagic.o >/dev/null 2>&1 mv -f .deps/ascmagic.Tpo .deps/ascmagic.Plo /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT compress.lo -MD -MP -MF .deps/compress.Tpo -c -o compress.lo compress.c gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT compress.lo -MD -MP -MF .deps/compress.Tpo -c compress.c -fno-common -DPIC -o .libs/compress.o gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT compress.lo -MD -MP -MF .deps/compress.Tpo -c compress.c -o compress.o >/dev/null 2>&1 mv -f .deps/compress.Tpo .deps/compress.Plo /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT is_tar.lo -MD -MP -MF .deps/is_tar.Tpo -c -o is_tar.lo is_tar.c gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT is_tar.lo -MD -MP -MF .deps/is_tar.Tpo -c is_tar.c -fno-common -DPIC -o .libs/is_tar.o gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT is_tar.lo -MD -MP -MF .deps/is_tar.Tpo -c is_tar.c -o is_tar.o >/dev/null 2>&1 mv -f .deps/is_tar.Tpo .deps/is_tar.Plo /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT readelf.lo -MD -MP -MF .deps/readelf.Tpo -c -o readelf.lo readelf.c gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT readelf.lo -MD -MP -MF .deps/readelf.Tpo -c readelf.c -fno-common -DPIC -o .libs/readelf.o gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT readelf.lo -MD -MP -MF .deps/readelf.Tpo -c readelf.c -o readelf.o >/dev/null 2>&1 mv -f .deps/readelf.Tpo .deps/readelf.Plo /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT print.lo -MD -MP -MF .deps/print.Tpo -c -o print.lo print.c gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT print.lo -MD -MP -MF .deps/print.Tpo -c print.c -fno-common -DPIC -o .libs/print.o gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT print.lo -MD -MP -MF .deps/print.Tpo -c print.c -o print.o >/dev/null 2>&1 mv -f .deps/print.Tpo .deps/print.Plo /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT fsmagic.lo -MD -MP -MF .deps/fsmagic.Tpo -c -o fsmagic.lo fsmagic.c gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT fsmagic.lo -MD -MP -MF .deps/fsmagic.Tpo -c fsmagic.c -fno-common -DPIC -o .libs/fsmagic.o gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT fsmagic.lo -MD -MP -MF .deps/fsmagic.Tpo -c fsmagic.c -o fsmagic.o >/dev/null 2>&1 mv -f .deps/fsmagic.Tpo .deps/fsmagic.Plo /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT funcs.lo -MD -MP -MF .deps/funcs.Tpo -c -o funcs.lo funcs.c gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT funcs.lo -MD -MP -MF .deps/funcs.Tpo -c funcs.c -fno-common -DPIC -o .libs/funcs.o gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT funcs.lo -MD -MP -MF .deps/funcs.Tpo -c funcs.c -o funcs.o >/dev/null 2>&1 mv -f .deps/funcs.Tpo .deps/funcs.Plo /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT apptype.lo -MD -MP -MF .deps/apptype.Tpo -c -o apptype.lo apptype.c gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT apptype.lo -MD -MP -MF .deps/apptype.Tpo -c apptype.c -fno-common -DPIC -o .libs/apptype.o gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -MT apptype.lo -MD -MP -MF .deps/apptype.Tpo -c apptype.c -o apptype.o >/dev/null 2>&1 mv -f .deps/apptype.Tpo .deps/apptype.Plo /bin/sh ../libtool --tag=CC --mode=link gcc -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra -Wunused-parameter -g -O2 -no-undefined -version-info 1:0:0 -o libmagic.la -rpath /usr/local/lib magic.lo apprentice.lo softmagic.lo ascmagic.lo compress.lo is_tar.lo readelf.lo print.lo fsmagic.lo funcs.lo apptype.lo -lz gcc -dynamiclib -o .libs/libmagic.1.0.0.dylib .libs/magic.o .libs/apprentice.o .libs/softmagic.o .libs/ascmagic.o .libs/compress.o .libs/is_tar.o .libs/readelf.o .libs/print.o .libs/fsmagic.o .libs/funcs.o .libs/apptype.o -lz -install_name /usr/local/lib/libmagic.1.dylib -Wl,-compatibility_version -Wl,2 -Wl,-current_version -Wl,2.0 ld: duplicate symbol _li in .libs/apprentice.o and .libs/magic.o collect2: ld returned 1 exit status make[2]: *** [libmagic.la] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 The only debugging step I know: macpro% nm src/.libs/apprentice.o | grep _li 0000cc40 S _li macpro% nm src/.libs/magic.o | grep _li 000029f8 S _li (NB: this symbol does not appear in either .o on my Fedora machine, although "li" (no underscore) appears in both, as a [C]ommon symbol) From kimmo at global-wire.fi Sun Aug 31 16:35:51 2008 From: kimmo at global-wire.fi (Kimmo Suominen) Date: Sun, 31 Aug 2008 16:35:51 +0300 Subject: 4.25 and 4.26 won't run on FC9; give spurious "magic version" error In-Reply-To: <48BA9C64.1030306@jay.fm> References: <48BA9C64.1030306@jay.fm> Message-ID: <20080831133551.GB5954@kimmo.suominen.com> What does the wrapper script do? Dynamic library configuration, e.g. with LD_* environment variables? What does ldd tell you about the binary? I'm guessing the binary gets linked with an old libmagic.so when run without the wrapper. Best regards, + Kimmo -- Kimmo Suominen On Sun, Aug 31, 2008 at 09:28:04AM -0400, Jay Levitt wrote: > Now I'm on a Fedora Core 9 box, x86_64. I can build both 4.25 and 4.26 > successfully, but both give the same odd error when run: > > [root at micro]/usr/local/src/tools/file-4.26/src/.libs# ./file -v > file-4.26 > magic file from /usr/local/share/file/magic > [root at micro]/usr/local/src/tools/file-4.26/src/.libs# ./file ./file > file: File 4.23 supports only 4 version magic files. > `/usr/local/share/file/magic.mgc' is version 6 > > Interestingly, the wrapper script DOES work... > > [root at micro]/usr/local/src/tools/file-4.26/src/.libs# cd .. > [root at micro]/usr/local/src/tools/file-4.26/src# ./file -v > lt-file-4.26 > magic file from /usr/local/share/file/magic > [root at micro]/usr/local/src/tools/file-4.26/src# ./file file > file: POSIX shell script text executable > > > I'm lost... > > Jay Levitt From lists-file at jay.fm Mon Sep 1 00:06:08 2008 From: lists-file at jay.fm (Jay Levitt) Date: Sun, 31 Aug 2008 17:06:08 -0400 Subject: 4.25 and 4.26 won't run on FC9; give spurious "magic version" error In-Reply-To: <20080831133551.GB5954@kimmo.suominen.com> References: <48BA9C64.1030306@jay.fm> <20080831133551.GB5954@kimmo.suominen.com> Message-ID: <48BB07C0.6010503@jay.fm> Kimmo Suominen wrote: > What does the wrapper script do? Dynamic library configuration, > e.g. with LD_* environment variables? elefino! It's part of the build process. But yes, it seems to: 1. Reset a bunch of compiler/linker paths, including PATH itself; 2. Check for $libtool_install_magic, and if it's set, if I'm reading right.. do.. nothing of consequence? (it sets notinst_deplibs, which isn't checked again) 3. Find itself 4. Rebuild file.o, linking with libmagic.so, if "necessary". FWIW, it's apparently not necessary on my system, because src/.libs/file hasn't been modified since yesterday 5. Run the program So it seems to me that, if running that wrapper works, then immediately running the executable should also work. But it doesn't. (And my brain hurts distinguishing $file from file from a file. Occupational hazard around here, I'm sure.) > What does ldd tell you about the binary? > > I'm guessing the binary gets linked with an old libmagic.so > when run without the wrapper. Oh, interesting.. it's linking to /usr/lib64/libmagic.so.1. The library we just built gets put in /usr/local/lib/, not /usr/local/lib64. ...and... typing ldconfig fixes it. Magic voodoo. But I don't get why... it looks like libtool should call ldconfig. Maybe libtool wasn't called at all? If you have any ideas, yell.. otherwise, if someone's interested, I'm happy to try to debug through the configure and makefile, but I can't for the life of me remember how to debug makefiles. Too much ruby! Which is, in fact, why I was upgrading to 4.26. Gotta love that. Jay Levitt From lists-file at jay.fm Fri Sep 12 16:30:40 2008 From: lists-file at jay.fm (Jay Levitt) Date: Fri, 12 Sep 2008 09:30:40 -0400 Subject: Can't build 4.26 on Mac OS X: duplicate symbol _li In-Reply-To: <48BA9A5B.50007@jay.fm> References: <48BA9A5B.50007@jay.fm> Message-ID: <48CA6F00.9060704@jay.fm> I hate to bump, but.. anyone? Any ideas? I'll volunteer to maintain the Mac port if I can get this working... Jay Levitt wrote: > I downloaded 4.26, but on my Mac (OS X 10.5.4 Leopard), I get a > duplicate symbol error from ld. I don't know enough about object > modules to debug unaided, but if someone can let me know how to > troubleshoot this, I'm happy to dig further. > 4.25 builds fine on the same machine; 4.26 builds on my Fedora machine > (but see my next post about strange problems running it there). I also > tried: > > make distclean > autoreconf > ./configure > make > > but the same thing happened. There's an interesting bug at > http://issues.sxemacs.org/show_bug.cgi?id=42 which may be related; > apparently Apple's gcc does something different with inlines and C99? > My C-fu is rusty (and predates C99). > > The error: > > ld: duplicate symbol _li in .libs/apprentice.o and .libs/magic.o > > Toolchain (copyrights snipped): > > macpro% gcc --version > i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5465) > macpro% ld -v > @(#)PROGRAM:ld PROJECT:ld64-77 > macpro% m4 --version > GNU M4 1.4.6 > macpro% make --version > GNU Make 3.81 > > The full configure/make output: > > macpro% ./configure > checking for a BSD-compatible install... /opt/local/bin/ginstall -c > checking whether build environment is sane... yes > checking for a thread-safe mkdir -p... /opt/local/bin/gmkdir -p > checking for gawk... gawk > checking whether make sets $(MAKE)... yes > checking for builtin ELF support... yes > checking for ELF core file support... yes > checking for file formats in man section 5... no > checking for gcc... gcc > checking for C compiler default output file name... a.out > checking whether the C compiler works... yes > checking whether we are cross compiling... no > checking for suffix of executables... > checking for suffix of object files... o > checking whether we are using the GNU C compiler... yes > checking whether gcc accepts -g... yes > checking for gcc option to accept ISO C89... none needed > checking for style of include used by make... GNU > checking dependency style of gcc... gcc3 > checking whether gcc and cc understand -c and -o together... yes > checking for a BSD-compatible install... /opt/local/bin/ginstall -c > checking whether ln -s works... yes > checking build system type... i386-apple-darwin9.4.0 > checking host system type... i386-apple-darwin9.4.0 > checking for a sed that does not truncate output... /usr/bin/sed > checking for grep that handles long lines and -e... /usr/bin/grep > checking for egrep... /usr/bin/grep -E > checking for ld used by gcc... /usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld > checking if the linker (/usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld) is > GNU ld... no > checking for /usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld option to > reload object files... -r > checking for BSD-compatible nm... /usr/bin/nm -p > checking how to recognise dependent libraries... pass_all > checking how to run the C preprocessor... gcc -E > checking for ANSI C header files... yes > checking for sys/types.h... yes > checking for sys/stat.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for memory.h... yes > checking for strings.h... yes > checking for inttypes.h... yes > checking for stdint.h... yes > checking for unistd.h... yes > checking dlfcn.h usability... yes > checking dlfcn.h presence... yes > checking for dlfcn.h... yes > checking for g++... g++ > checking whether we are using the GNU C++ compiler... yes > checking whether g++ accepts -g... yes > checking dependency style of g++... gcc3 > checking how to run the C++ preprocessor... g++ -E > checking for g77... no > checking for xlf... no > checking for f77... no > checking for frt... no > checking for pgf77... no > checking for cf77... no > checking for fort77... no > checking for fl32... no > checking for af77... no > checking for xlf90... no > checking for f90... no > checking for pgf90... no > checking for pghpf... no > checking for epcf90... no > checking for gfortran... no > checking for g95... no > checking for xlf95... no > checking for f95... no > checking for fort... no > checking for ifort... no > checking for ifc... no > checking for efc... no > checking for pgf95... no > checking for lf95... no > checking for ftn... no > checking whether we are using the GNU Fortran 77 compiler... no > checking whether accepts -g... no > checking the maximum length of command line arguments... 196608 > checking command to parse /usr/bin/nm -p output from gcc object... rm: > conftest.dSYM: is a directory > rm: conftest.dSYM: is a directory > rm: conftest.dSYM: is a directory > rm: conftest.dSYM: is a directory > ok > checking for objdir... .libs > checking for ar... ar > checking for ranlib... ranlib > checking for strip... strip > rm: conftest.dSYM: is a directory > rm: conftest.dSYM: is a directory > checking if gcc static flag works... rm: conftest.dSYM: is a directory > yes > checking if gcc supports -fno-rtti -fno-exceptions... rm: conftest.dSYM: > is a directory > no > checking for gcc option to produce PIC... -fno-common > checking if gcc PIC flag -fno-common works... rm: conftest.dSYM: is a > directory > yes > checking if gcc supports -c -o file.o... rm: conftest.dSYM: is a directory > yes > checking whether the gcc linker > (/usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld) supports shared > libraries... yes > checking dynamic linker characteristics... darwin9.4.0 dyld > checking how to hardcode library paths into programs... immediate > checking whether stripping libraries is possible... yes > checking if libtool supports shared libraries... yes > checking whether to build shared libraries... yes > checking whether to build static libraries... yes > configure: creating libtool > appending configuration tag "CXX" to libtool > rm: conftest.dSYM: is a directory > rm: conftest.dSYM: is a directory > checking for ld used by g++... /usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld > checking if the linker (/usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld) is > GNU ld... no > checking whether the g++ linker > (/usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld) supports shared > libraries... yes > checking for g++ option to produce PIC... -fno-common > checking if g++ PIC flag -fno-common works... rm: conftest.dSYM: is a > directory > yes > checking if g++ supports -c -o file.o... rm: conftest.dSYM: is a directory > yes > checking whether the g++ linker > (/usr/libexec/gcc/i686-apple-darwin9/4.0.1/ld) supports shared > libraries... yes > checking dynamic linker characteristics... darwin9.4.0 dyld > checking how to hardcode library paths into programs... immediate > checking whether stripping libraries is possible... yes > appending configuration tag "F77" to libtool > checking for ANSI C header files... (cached) yes > checking whether sys/types.h defines makedev... yes > checking for sys/wait.h that is POSIX.1 compatible... yes > checking for stdint.h... (cached) yes > checking fcntl.h usability... yes > checking fcntl.h presence... yes > checking for fcntl.h... yes > checking locale.h usability... yes > checking locale.h presence... yes > checking for locale.h... yes > checking for stdint.h... (cached) yes > checking for inttypes.h... (cached) yes > checking for unistd.h... (cached) yes > checking utime.h usability... yes > checking utime.h presence... yes > checking for utime.h... yes > checking wchar.h usability... yes > checking wchar.h presence... yes > checking for wchar.h... yes > checking wctype.h usability... yes > checking wctype.h presence... yes > checking for wctype.h... yes > checking limits.h usability... yes > checking limits.h presence... yes > checking for limits.h... yes > checking getopt.h usability... yes > checking getopt.h presence... yes > checking for getopt.h... yes > checking err.h usability... yes > checking err.h presence... yes > checking for err.h... yes > checking sys/mman.h usability... yes > checking sys/mman.h presence... yes > checking for sys/mman.h... yes > checking for sys/stat.h... (cached) yes > checking for sys/types.h... (cached) yes > checking sys/utime.h usability... no > checking sys/utime.h presence... no > checking for sys/utime.h... no > checking sys/time.h usability... yes > checking sys/time.h presence... yes > checking for sys/time.h... yes > checking zlib.h usability... yes > checking zlib.h presence... yes > checking for zlib.h... yes > checking for an ANSI C-conforming const... yes > checking for off_t... yes > checking for size_t... yes > checking for struct stat.st_rdev... yes > checking whether struct tm is in sys/time.h or time.h... time.h > checking for tm_zone in struct tm... yes > checking for tzname... yes > checking for tm_isdst in struct tm... yes > checking for daylight... yes > checking for special C compiler options needed for large files... no > checking for _FILE_OFFSET_BITS value needed for large files... no > rm: conftest.dSYM: is a directory > checking for _LARGEFILE_SOURCE value needed for large files... no > rm: conftest.dSYM: is a directory > checking for mbstate_t... yes > checking for uint8_t... yes > checking for uint16_t... yes > checking for uint32_t... yes > checking for int32_t... yes > checking for uint64_t... yes > checking for int64_t... yes > checking for long long... yes > checking size of long long... 8 > checking for gcc compiler warnings... yes > checking for mmap... yes > checking for strerror... yes > checking for strndup... no > checking for strtoul... yes > checking for mbrtowc... yes > checking for mkstemp... yes > checking for utimes... yes > checking for utime... yes > checking for wcwidth... yes > checking for strtof... yes > checking for getopt_long... yes > checking for asprintf... yes > checking for vasprintf... yes > checking for gzopen in -lz... yes > configure: creating ./config.status > config.status: creating Makefile > config.status: creating src/Makefile > config.status: creating magic/Makefile > config.status: creating tests/Makefile > config.status: creating doc/Makefile > config.status: creating python/Makefile > config.status: creating config.h > config.status: config.h is unchanged > config.status: executing depfiles commands > macpro% make > make all-recursive > Making all in src > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT magic.lo -MD -MP -MF .deps/magic.Tpo -c -o > magic.lo magic.c > mkdir .libs > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT magic.lo -MD -MP -MF .deps/magic.Tpo -c > magic.c -fno-common -DPIC -o .libs/magic.o > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT magic.lo -MD -MP -MF .deps/magic.Tpo -c > magic.c -o magic.o >/dev/null 2>&1 > mv -f .deps/magic.Tpo .deps/magic.Plo > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT apprentice.lo -MD -MP -MF > .deps/apprentice.Tpo -c -o apprentice.lo apprentice.c > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT apprentice.lo -MD -MP -MF > .deps/apprentice.Tpo -c apprentice.c -fno-common -DPIC -o > .libs/apprentice.o > apprentice.c: In function 'load_1': > apprentice.c:1699: warning: 'slen' may be used uninitialized in this > function > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT apprentice.lo -MD -MP -MF > .deps/apprentice.Tpo -c apprentice.c -o apprentice.o >/dev/null 2>&1 > mv -f .deps/apprentice.Tpo .deps/apprentice.Plo > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT softmagic.lo -MD -MP -MF > .deps/softmagic.Tpo -c -o softmagic.lo softmagic.c > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT softmagic.lo -MD -MP -MF > .deps/softmagic.Tpo -c softmagic.c -fno-common -DPIC -o .libs/softmagic.o > softmagic.c: In function 'mcopy': > softmagic.c:848: warning: format '%zu' expects type 'size_t', but > argument 3 has type 'uint32_t' > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT softmagic.lo -MD -MP -MF > .deps/softmagic.Tpo -c softmagic.c -o softmagic.o >/dev/null 2>&1 > mv -f .deps/softmagic.Tpo .deps/softmagic.Plo > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT ascmagic.lo -MD -MP -MF .deps/ascmagic.Tpo > -c -o ascmagic.lo ascmagic.c > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT ascmagic.lo -MD -MP -MF .deps/ascmagic.Tpo > -c ascmagic.c -fno-common -DPIC -o .libs/ascmagic.o > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT ascmagic.lo -MD -MP -MF .deps/ascmagic.Tpo > -c ascmagic.c -o ascmagic.o >/dev/null 2>&1 > mv -f .deps/ascmagic.Tpo .deps/ascmagic.Plo > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT compress.lo -MD -MP -MF .deps/compress.Tpo > -c -o compress.lo compress.c > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT compress.lo -MD -MP -MF .deps/compress.Tpo > -c compress.c -fno-common -DPIC -o .libs/compress.o > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT compress.lo -MD -MP -MF .deps/compress.Tpo > -c compress.c -o compress.o >/dev/null 2>&1 > mv -f .deps/compress.Tpo .deps/compress.Plo > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT is_tar.lo -MD -MP -MF .deps/is_tar.Tpo -c > -o is_tar.lo is_tar.c > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT is_tar.lo -MD -MP -MF .deps/is_tar.Tpo -c > is_tar.c -fno-common -DPIC -o .libs/is_tar.o > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT is_tar.lo -MD -MP -MF .deps/is_tar.Tpo -c > is_tar.c -o is_tar.o >/dev/null 2>&1 > mv -f .deps/is_tar.Tpo .deps/is_tar.Plo > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT readelf.lo -MD -MP -MF .deps/readelf.Tpo > -c -o readelf.lo readelf.c > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT readelf.lo -MD -MP -MF .deps/readelf.Tpo > -c readelf.c -fno-common -DPIC -o .libs/readelf.o > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT readelf.lo -MD -MP -MF .deps/readelf.Tpo > -c readelf.c -o readelf.o >/dev/null 2>&1 > mv -f .deps/readelf.Tpo .deps/readelf.Plo > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT print.lo -MD -MP -MF .deps/print.Tpo -c -o > print.lo print.c > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT print.lo -MD -MP -MF .deps/print.Tpo -c > print.c -fno-common -DPIC -o .libs/print.o > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT print.lo -MD -MP -MF .deps/print.Tpo -c > print.c -o print.o >/dev/null 2>&1 > mv -f .deps/print.Tpo .deps/print.Plo > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT fsmagic.lo -MD -MP -MF .deps/fsmagic.Tpo > -c -o fsmagic.lo fsmagic.c > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT fsmagic.lo -MD -MP -MF .deps/fsmagic.Tpo > -c fsmagic.c -fno-common -DPIC -o .libs/fsmagic.o > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT fsmagic.lo -MD -MP -MF .deps/fsmagic.Tpo > -c fsmagic.c -o fsmagic.o >/dev/null 2>&1 > mv -f .deps/fsmagic.Tpo .deps/fsmagic.Plo > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT funcs.lo -MD -MP -MF .deps/funcs.Tpo -c -o > funcs.lo funcs.c > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT funcs.lo -MD -MP -MF .deps/funcs.Tpo -c > funcs.c -fno-common -DPIC -o .libs/funcs.o > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT funcs.lo -MD -MP -MF .deps/funcs.Tpo -c > funcs.c -o funcs.o >/dev/null 2>&1 > mv -f .deps/funcs.Tpo .deps/funcs.Plo > /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. > -I.. -DMAGIC='"/usr/local/share/file/magic"' -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT apptype.lo -MD -MP -MF .deps/apptype.Tpo > -c -o apptype.lo apptype.c > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT apptype.lo -MD -MP -MF .deps/apptype.Tpo > -c apptype.c -fno-common -DPIC -o .libs/apptype.o > gcc -DHAVE_CONFIG_H -I. -I.. -DMAGIC=\"/usr/local/share/file/magic\" > -Wall -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith > -Wmissing-declarations -Wredundant-decls -Wnested-externs -Wsign-compare > -Wreturn-type -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -MT apptype.lo -MD -MP -MF .deps/apptype.Tpo > -c apptype.c -o apptype.o >/dev/null 2>&1 > mv -f .deps/apptype.Tpo .deps/apptype.Plo > /bin/sh ../libtool --tag=CC --mode=link gcc -Wall -Wstrict-prototypes > -Wmissing-prototypes -Wpointer-arith -Wmissing-declarations > -Wredundant-decls -Wnested-externs -Wsign-compare -Wreturn-type > -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wextra > -Wunused-parameter -g -O2 -no-undefined -version-info 1:0:0 -o > libmagic.la -rpath /usr/local/lib magic.lo apprentice.lo softmagic.lo > ascmagic.lo compress.lo is_tar.lo readelf.lo print.lo fsmagic.lo > funcs.lo apptype.lo -lz > gcc -dynamiclib -o .libs/libmagic.1.0.0.dylib .libs/magic.o > .libs/apprentice.o .libs/softmagic.o .libs/ascmagic.o .libs/compress.o > .libs/is_tar.o .libs/readelf.o .libs/print.o .libs/fsmagic.o > .libs/funcs.o .libs/apptype.o -lz -install_name > /usr/local/lib/libmagic.1.dylib -Wl,-compatibility_version -Wl,2 > -Wl,-current_version -Wl,2.0 > ld: duplicate symbol _li in .libs/apprentice.o and .libs/magic.o > > collect2: ld returned 1 exit status > make[2]: *** [libmagic.la] Error 1 > make[1]: *** [all-recursive] Error 1 > make: *** [all] Error 2 > > The only debugging step I know: > > macpro% nm src/.libs/apprentice.o | grep _li > 0000cc40 S _li > macpro% nm src/.libs/magic.o | grep _li > 000029f8 S _li > > (NB: this symbol does not appear in either .o on my Fedora machine, > although "li" (no underscore) appears in both, as a [C]ommon symbol) > > _______________________________________________ > File mailing list > File at mx.gw.com > http://mx.gw.com/mailman/listinfo/file From christos at zoulas.com Fri Sep 12 17:01:45 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 12 Sep 2008 10:01:45 -0400 Subject: Can't build 4.26 on Mac OS X: duplicate symbol _li In-Reply-To: <48CA6F00.9060704@jay.fm> from Jay Levitt (Sep 12, 9:30am) Message-ID: <20080912140145.1C6055654E@rebar.astron.com> On Sep 12, 9:30am, lists-file at jay.fm (Jay Levitt) wrote: -- Subject: Re: Can't build 4.26 on Mac OS X: duplicate symbol _li | I hate to bump, but.. anyone? Any ideas? I'll volunteer to maintain the | Mac port if I can get this working... There you go :-) I h8 darwin's mach-o crap. christos Index: file.h =================================================================== RCS file: /p/file/cvsroot/file/src/file.h,v retrieving revision 1.108 retrieving revision 1.109 diff -u -r1.108 -r1.109 --- file.h 16 Jul 2008 18:00:57 -0000 1.108 +++ file.h 2 Sep 2008 07:11:30 -0000 1.109 @@ -27,7 +27,7 @@ */ /* * file.h - definitions for file(1) program - * @(#)$File: file.h,v 1.108 2008/07/16 18:00:57 christos Exp $ + * @(#)$File: file.h,v 1.109 2008/09/02 07:11:30 christos Exp $ */ #ifndef __file_h__ @@ -302,7 +302,7 @@ int last_match; int last_cond; /* used for error checking by parse() */ #endif -} *li; +}; struct magic_set { struct mlist *mlist; struct cont { From Joerg-Cyril.Hoehle at t-systems.com Wed Oct 1 16:24:50 2008 From: Joerg-Cyril.Hoehle at t-systems.com (Hoehle, Joerg-Cyril) Date: Wed, 1 Oct 2008 15:24:50 +0200 Subject: ~/.magic prevents use of global /usr/share/file/magic Message-ID: <47CC5ABB01651443A88DB8EC5B4D657B0186D909@S4DE8PSAANK.mitte.t-com.de> Hi, (observed in file 4.21) When a ~/.magic file is present, the central magic database in /usr/share/file/* is not consulted anymore. As a result, "file foobar" only knows about the few formats defined in the user's ~/.magic. This is both inconvenient and contrary to the manpage, which says: "In addition file will look in ... $HOME/.magic for magic entries." OTOH, using /etc/magic still works *in addition* to /usr/share/file/. This may be related to the following observation: http://mx.gw.com/pipermail/file/2008/000235.html >At the moment, when using magic_load (libmagic(3)), if the file name used >as the second parameter is something other than NULL, then ONLY that file >is used (and not the default database file). See also Ubuntu bug https://bugs.launchpad.net/bugs/259851 Regards, J?rg H?hle From Joerg-Cyril.Hoehle at t-systems.com Wed Oct 1 16:16:37 2008 From: Joerg-Cyril.Hoehle at t-systems.com (Hoehle, Joerg-Cyril) Date: Wed, 1 Oct 2008 15:16:37 +0200 Subject: WG: magic entry for BINK video files Message-ID: <47CC5ABB01651443A88DB8EC5B4D657B0186D8FC@S4DE8PSAANK.mitte.t-com.de> Dear file&magic maintainers, here's an entry for the BINK video format. This format is used by many PC games, which are now playable on Linux & Intel Macs thanks to the wine project. I wrote this description based on information from the following sources: http://www.radgametools.com/bnkmain.htm tools http://wiki.multimedia.cx/index.php?title=Bink_Container # Magic local data for file(1) command. # Insert here your local magic data. Format is described in magic(5). # Type: Bink Video # URL: http://wiki.multimedia.cx/index.php?title=Bink_Container # From: 2008-07-18 0 string BIK Bink Video >3 regex =[a-z] rev.%s #>4 ulelong x size %d >20 ulelong x \b, %d >24 ulelong x \bx%d >8 ulelong x \b, %d frames >32 ulelong x at rate %d/ >28 ulelong >1 \b%d >40 ulelong =0 \b, no audio >40 ulelong !0 \b, %d audio track >>40 ulelong !1 \bs # follow properties of the first audio track only >>48 uleshort x %dHz >>51 byte&0x20 0 mono >>51 byte&0x20 !0 stereo #>>51 byte&0x10 0 FFT #>>51 byte&0x10 !0 DCT Some sample output: intro_big.bik: Bink Video rev.f, 560x333, 3282 frames at rate 1/24, 1 audio track 22050Hz stereo FFT Intro.bik: Bink Video rev.i, 640x368, 1214 frames at rate 1/15, 2 audio tracks 3584Hz mono FFT CI12535-01.bik: Bink Video rev.i, 799x599, 75 frames at rate 1/25, 1 audio track 44100Hz mono DCT CIN40003_Fin.bik: Bink Video rev.i, 800x432, 4035 frames at rate 1/25, 1 audio track 44100Hz stereo DCT CI40084-00.bik: Bink Video rev.i, 800x600, 206 frames at rate 1/25, no audio I later commented out the FFT/DCT parts which are not self-explanatory or particularly worthwhile. About using lelong: I don't know how BINK cares about endianness. All files I have originate from MS-Windows Intel PC, so little endian is used. Regards, J?rg H?hle -------------- next part -------------- A non-text attachment was scrubbed... Name: magic Type: application/octet-stream Size: 674 bytes Desc: magic URL: From christos at zoulas.com Wed Oct 1 17:28:42 2008 From: christos at zoulas.com (Christos Zoulas) Date: Wed, 1 Oct 2008 10:28:42 -0400 Subject: WG: magic entry for BINK video files In-Reply-To: <47CC5ABB01651443A88DB8EC5B4D657B0186D8FC@S4DE8PSAANK.mitte.t-com.de> from "Hoehle, Joerg-Cyril" (Oct 1, 3:16pm) Message-ID: <20081001142842.DB22F5654E@rebar.astron.com> On Oct 1, 3:16pm, Joerg-Cyril.Hoehle at t-systems.com ("Hoehle, Joerg-Cyril") wrote: -- Subject: WG: magic entry for BINK video files | Dear file&magic maintainers, | | here's an entry for the BINK video format. This format is used by many PC g= | ames, which are now playable on Linux & Intel Macs thanks to the wine proje= | ct. I wrote this description based on information from the following source= | s: | http://www.radgametools.com/bnkmain.htm tools | http://wiki.multimedia.cx/index.php?title=3DBink_Container | | # Magic local data for file(1) command. | # Insert here your local magic data. Format is described in magic(5). | | # Type: Bink Video | # URL: http://wiki.multimedia.cx/index.php?title=3DBink_Container | # From: 2008-07-18 | | 0 string BIK Bink Video | >3 regex =3D[a-z] rev.%s | #>4 ulelong x size %d | >20 ulelong x \b, %d | >24 ulelong x \bx%d | >8 ulelong x \b, %d frames | >32 ulelong x at rate %d/ | >28 ulelong >1 \b%d | >40 ulelong =3D0 \b, no audio | >40 ulelong !0 \b, %d audio track | >>40 ulelong !1 \bs | # follow properties of the first audio track only | >>48 uleshort x %dHz | >>51 byte&0x20 0 mono | >>51 byte&0x20 !0 stereo | #>>51 byte&0x10 0 FFT | #>>51 byte&0x10 !0 DCT | | Some sample output: | intro_big.bik: Bink Video rev.f, 560x333, 3282 frames at rate 1/24, 1 a= | udio track 22050Hz stereo FFT | Intro.bik: Bink Video rev.i, 640x368, 1214 frames at rate 1/15, 2 a= | udio tracks 3584Hz mono FFT | CI12535-01.bik: Bink Video rev.i, 799x599, 75 frames at rate 1/25, 1 aud= | io track 44100Hz mono DCT | CIN40003_Fin.bik: Bink Video rev.i, 800x432, 4035 frames at rate 1/25, 1 a= | udio track 44100Hz stereo DCT | CI40084-00.bik: Bink Video rev.i, 800x600, 206 frames at rate 1/25, no a= | udio | | I later commented out the FFT/DCT parts which are not self-explanatory or p= | articularly worthwhile. | | About using lelong: I don't know how BINK cares about endianness. | All files I have originate from MS-Windows Intel PC, so little endian | is used. Thanks a lot. christos From christos at zoulas.com Wed Oct 1 17:29:30 2008 From: christos at zoulas.com (Christos Zoulas) Date: Wed, 1 Oct 2008 10:29:30 -0400 Subject: ~/.magic prevents use of global /usr/share/file/magic In-Reply-To: <47CC5ABB01651443A88DB8EC5B4D657B0186D909@S4DE8PSAANK.mitte.t-com.de> from "Hoehle, Joerg-Cyril" (Oct 1, 3:24pm) Message-ID: <20081001142930.5F01556550@rebar.astron.com> On Oct 1, 3:24pm, Joerg-Cyril.Hoehle at t-systems.com ("Hoehle, Joerg-Cyril") wrote: -- Subject: ~/.magic prevents use of global /usr/share/file/magic | Hi, | | (observed in file 4.21) | | When a ~/.magic file is present, the central magic database in /usr/share/f= | ile/* is not consulted anymore. As a result, "file foobar" only knows about= | the few formats defined in the user's ~/.magic. | | This is both inconvenient and contrary to the manpage, which says: "In addi= | tion file will look in ... $HOME/.magic for magic entries." | OTOH, using /etc/magic still works *in addition* to /usr/share/file/. | | This may be related to the following observation: | http://mx.gw.com/pipermail/file/2008/000235.html | >At the moment, when using magic_load (libmagic(3)), if the file name used | >as the second parameter is something other than NULL, then ONLY that file | >is used (and not the default database file). | | See also Ubuntu bug https://bugs.launchpad.net/bugs/259851 | Index: file.c =================================================================== RCS file: /p/file/cvsroot/file/src/file.c,v retrieving revision 1.121 diff -u -u -r1.121 file.c --- file.c 3 Jul 2008 15:48:18 -0000 1.121 +++ file.c 1 Oct 2008 14:29:04 -0000 @@ -87,7 +87,7 @@ # define USAGE "Usage: %s [-bcik" SYMLINKFLAG "nNrsvz0] [-e test] [-f namefile] [-F separator] [-m magicfiles] file...\n %s -C -m magicfiles\n" #ifndef MAXPATHLEN -#define MAXPATHLEN 512 +#define MAXPATHLEN 1024 #endif private int /* Global command-line options */ @@ -96,7 +96,7 @@ nobuffer = 0, /* Do not buffer stdout */ nulsep = 0; /* Append '\0' to the separator */ -private const char *magicfile = 0; /* where the magic is */ +private const char *magicfile; /* where the magic is */ private const char *default_magicfile = MAGIC; private const char *separator = ":"; /* Default field separator */ @@ -124,8 +124,8 @@ int action = 0, didsomefiles = 0, errflg = 0; int flags = 0; char *home, *usermagic; - struct stat sb; static const char hmagic[] = "/.magic"; + char magicpath[2 * MAXPATHLEN + 2]; #define OPTSTRING "bcCde:f:F:hikLm:nNprsvz0" int longindex; static const struct option long_options[] = @@ -171,14 +171,12 @@ magicfile = usermagic; else if ((home = getenv("HOME")) != NULL) { - if ((usermagic = malloc(strlen(home) - + sizeof(hmagic))) != NULL) { - (void)strcpy(usermagic, home); - (void)strcat(usermagic, hmagic); - if (stat(usermagic, &sb)<0) - free(usermagic); - else - magicfile = usermagic; + (void)snprintf(magicpath, sizeof(magicpath), "%s%s", + home, hmagic); + if (access(magicpath, R_OK) == 0) { + (void)snprintf(magicpath, sizeof(magicpath), + "%s%s:%s", home, hmagic, magicfile); + magicfile = magicpath; } } @@ -293,6 +291,12 @@ switch(action) { case FILE_CHECK: case FILE_COMPILE: + /* + * Don't try to check/compile ~/.magic unless we explicitly + * ask for it. + */ + if (magicfile == magicpath) + magicfile = default_magicfile; magic = magic_open(flags|MAGIC_CHECK); if (magic == NULL) { (void)fprintf(stderr, "%s: %s\n", progname, From dnovotny at redhat.com Fri Oct 3 15:51:41 2008 From: dnovotny at redhat.com (Daniel Novotny) Date: Fri, 3 Oct 2008 08:51:41 -0400 (EDT) Subject: recognition of MIME types (-i) of Microsoft Office files In-Reply-To: <1166815789.51381223038073753.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <845091101.52061223038300998.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> hello, I have a question: if I want to distinguish between MIME types of different MS office formats, is there a way to do so, or do you plan to support MIME types such as application/msword and application/powerpoint ? they can be opened by different applications, so there is a reason to distinguish them. the latest file(1) version goes like this: [danny at localhost file]$ file testf* testf.doc: OLE 2 Compound Document testf.ppt: OLE 2 Compound Document testf.xls: OLE 2 Compound Document [danny at localhost file]$ file -i testf* testf.doc: application/octet-stream testf.ppt: application/octet-stream testf.xls: application/octet-stream [danny at localhost file]$ the version 4.10 from RHEL4 says "application/msword" for all three cases... another point: -in this web page the guy says he relies on extension, but this is very non-UNIX, bc the file(1) utility is "above" extensions, transcending them: http://www.mediawiki.org/wiki/Manual_talk:Mime_type_detection -there is a text P\0w\0\e\0\r\0p\o\0i\0\n\0t in the power point document, but it might not be in the first HOWMANY bytes of the file so 0 search/0xFFFFF \120\000\157\000\167\000\145\000\162\000\120\000\157\000\151\000\156\000\164\000 application/powerpoint may or may not work maybe we can do a special case, as with elf binaries or text files... to look to entire file when office magic appears in the header ... any ideas how to do this without breaking the structure of the entire file(1) program? best regards, Daniel Novotny From christos at zoulas.com Fri Oct 3 17:49:22 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 3 Oct 2008 10:49:22 -0400 Subject: recognition of MIME types (-i) of Microsoft Office files In-Reply-To: <845091101.52061223038300998.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> from Daniel Novotny (Oct 3, 8:51am) Message-ID: <20081003144922.3A31356550@rebar.astron.com> On Oct 3, 8:51am, dnovotny at redhat.com (Daniel Novotny) wrote: -- Subject: recognition of MIME types (-i) of Microsoft Office files | hello, | | I have a question: if I want to distinguish between MIME types of different | MS office formats, is there a way to do so, or do you plan to support | MIME types such as application/msword and application/powerpoint ? | they can be opened by different applications, so there is a reason | to distinguish them. | | the latest file(1) version goes like this: | | [danny at localhost file]$ file testf* | testf.doc: OLE 2 Compound Document | testf.ppt: OLE 2 Compound Document | testf.xls: OLE 2 Compound Document | [danny at localhost file]$ file -i testf* | testf.doc: application/octet-stream | testf.ppt: application/octet-stream | testf.xls: application/octet-stream | [danny at localhost file]$ | | the version 4.10 from RHEL4 says "application/msword" for all three cases... | | another point: | | -in this web page the guy says he relies on extension, but this is very non-UNIX, bc | the file(1) utility is "above" extensions, transcending them: | http://www.mediawiki.org/wiki/Manual_talk:Mime_type_detection | | -there is a text P\0w\0\e\0\r\0p\o\0i\0\n\0t in the power point | document, but it might not be in the first HOWMANY bytes of the file | | so | 0 search/0xFFFFF | \120\000\157\000\167\000\145\000\162\000\120\000\157\000\151\000\156\000\164\000 | application/powerpoint | | may or may not work | | maybe we can do a special case, as with elf binaries or text files... to look to entire file when | office magic appears in the header ... any ideas how to do this without breaking the | structure of the entire file(1) program? We need to write a special handler for OLE documents in C so that we can handle all the office formats. There is no good pattern to search/match. christos From randy at linuxfromscratch.org Thu Oct 9 11:04:25 2008 From: randy at linuxfromscratch.org (Randy McMurchy) Date: Thu, 09 Oct 2008 03:04:25 -0500 Subject: Problems with 'file --exclude troff' Message-ID: <48EDBB09.2060109@linuxfromscratch.org> Hi all, I'm new to this list but I searched the archives for anything that might be related to the issue I'm about to describe and couldn't find anything so I thought I'd bring it up here. It was brought to my attention that sometime after the 4.23 release the -e (--exclude) parameter did not work for the 'troff' test. Using 4.26, I looked into into the code and found the following: Using the output from the man page, there are 3 -e tests that don't work. The 'fortran', 'token', and 'troff' tests. Seems to me looking through the ChangeLog that the 'fortran' test was converted into a 'soft' test, so that explains why that one wouldn't work (though the man page could use updating). The 'token' test fails, because the man page actually identifies it incorrectly and it should be 'tokens' instead. I can't figure out the 'troff' test as there isn't anything in any of the ChangeLogs that show that it was intentionally removed. Anyway, I created this patch (which probably is wrong because I re-enabled the fortran test), but it does fix the 'tokens' and 'troff' tests. Please let me know if indeed the 'troff' exclude test was intentionally removed, or if it had been done by mistake somehow. Patch follows: diff -Naur file-orig/doc/file.man file-4.26/doc/file.man --- file-orig/doc/file.man 2008-03-07 15:00:07.000000000 +0000 +++ file-4.26/doc/file.man 2008-10-08 13:57:37.000000000 +0000 @@ -192,7 +192,7 @@ Don't consult magic files. .It tar Don't examine tar files. -.It token +.It tokens Don't look for known tokens inside ascii files. .It troff Don't look for troff sequences inside ascii files. diff -Naur file-orig/src/file.c file-4.26/src/file.c --- file-orig/src/file.c 2008-07-03 15:48:18.000000000 +0000 +++ file-4.26/src/file.c 2008-10-08 13:59:14.000000000 +0000 @@ -148,9 +148,11 @@ { "ascii", MAGIC_NO_CHECK_ASCII }, { "compress", MAGIC_NO_CHECK_COMPRESS }, { "elf", MAGIC_NO_CHECK_ELF }, + { "fortran", MAGIC_NO_CHECK_FORTRAN }, { "soft", MAGIC_NO_CHECK_SOFT }, { "tar", MAGIC_NO_CHECK_TAR }, { "tokens", MAGIC_NO_CHECK_TOKENS }, + { "troff", MAGIC_NO_CHECK_TROFF }, }; /* makes islower etc work for other langs */ -- Randy From lists-file at jay.fm Thu Oct 9 16:18:46 2008 From: lists-file at jay.fm (Jay Levitt) Date: Thu, 09 Oct 2008 09:18:46 -0400 Subject: Can't build 4.26 on Mac OS X: duplicate symbol _li In-Reply-To: <20080912140145.1C6055654E@rebar.astron.com> References: <20080912140145.1C6055654E@rebar.astron.com> Message-ID: <48EE04B6.6070809@jay.fm> Christos Zoulas wrote: > On Sep 12, 9:30am, lists-file at jay.fm (Jay Levitt) wrote: > -- Subject: Re: Can't build 4.26 on Mac OS X: duplicate symbol _li > > | I hate to bump, but.. anyone? Any ideas? I'll volunteer to maintain the > | Mac port if I can get this working... > > There you go :-) I h8 darwin's mach-o crap. Belated thanks! file 4.26 is now in MacPorts. Jay From christos at zoulas.com Thu Oct 9 20:26:24 2008 From: christos at zoulas.com (Christos Zoulas) Date: Thu, 9 Oct 2008 13:26:24 -0400 Subject: Problems with 'file --exclude troff' In-Reply-To: <48EDBB09.2060109@linuxfromscratch.org> from Randy McMurchy (Oct 9, 3:04am) Message-ID: <20081009172624.585365654E@rebar.astron.com> On Oct 9, 3:04am, randy at linuxfromscratch.org (Randy McMurchy) wrote: -- Subject: Problems with 'file --exclude troff' | Hi all, | | I'm new to this list but I searched the archives for | anything that might be related to the issue I'm about | to describe and couldn't find anything so I thought | I'd bring it up here. | | It was brought to my attention that sometime after | the 4.23 release the -e (--exclude) parameter did not | work for the 'troff' test. Using 4.26, I looked into | into the code and found the following: | | Using the output from the man page, there are 3 -e | tests that don't work. The 'fortran', 'token', and | 'troff' tests. Seems to me looking through the ChangeLog | that the 'fortran' test was converted into a 'soft' | test, so that explains why that one wouldn't work | (though the man page could use updating). | | The 'token' test fails, because the man page actually | identifies it incorrectly and it should be 'tokens' | instead. I can't figure out the 'troff' test as there | isn't anything in any of the ChangeLogs that show that | it was intentionally removed. | | Anyway, I created this patch (which probably is wrong | because I re-enabled the fortran test), but it does | fix the 'tokens' and 'troff' tests. | | Please let me know if indeed the 'troff' exclude test | was intentionally removed, or if it had been done by | mistake somehow. | | Patch follows: Well, both the fortran test and the troff test have been converted to soft tests. I have removed them from the doc and changed token to tokens. Thanks, christos From dnovotny at redhat.com Mon Oct 13 14:56:48 2008 From: dnovotny at redhat.com (Daniel Novotny) Date: Mon, 13 Oct 2008 07:56:48 -0400 (EDT) Subject: a patch for a crash in a .xlx file In-Reply-To: <1500268147.15091223898932948.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <1497677797.15121223899008724.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Hello, we have run into a problem with one specific file which caused a "vasprintf failed" crash: http://people.fedoraproject.org/~dnovotny/miXart8.xlx was incorrectly classified as a "DOS executable (device driver)", because it begins with 0xffffffff and has further characteristics, according to "magic/Magdir/msdos" magicfile "file" tries to print the "device driver name" and fails, returning error to the parent process, making domino effect in rpmbuild(8) my patch defines the "DOS executable (device driver)" more precisely, so there is no "string x" ("any" string) in the magic, but the specific names of the specific device drivers instead. this way the crash is averted. patch attached best, Daniel Novotny -------------- next part -------------- A non-text attachment was scrubbed... Name: file-4.26-devdrv.patch Type: text/x-patch Size: 1379 bytes Desc: not available URL: From christos at zoulas.com Mon Oct 13 15:53:26 2008 From: christos at zoulas.com (Christos Zoulas) Date: Mon, 13 Oct 2008 08:53:26 -0400 Subject: a patch for a crash in a .xlx file In-Reply-To: <1497677797.15121223899008724.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> from Daniel Novotny (Oct 13, 7:56am) Message-ID: <20081013125327.094715654E@rebar.astron.com> On Oct 13, 7:56am, dnovotny at redhat.com (Daniel Novotny) wrote: -- Subject: a patch for a crash in a .xlx file | Hello, | | we have run into a problem with one specific file which caused a "vasprintf failed" crash: | | http://people.fedoraproject.org/~dnovotny/miXart8.xlx was incorrectly classified as a "DOS executable (device driver)", | because it begins with 0xffffffff and has further characteristics, according to "magic/Magdir/msdos" magicfile | | "file" tries to print the "device driver name" and fails, returning error to the parent process, making domino effect in rpmbuild(8) | | my patch defines the "DOS executable (device driver)" more precisely, so there is no "string x" | ("any" string) in the magic, but the specific names of the specific device drivers instead. this way the crash is averted. | | patch attached I am not sure that hard-coding the driver names is the besit way of going about it and the guy that wrote most of thisd stuff has made a lot of changes making your patch not to apply cleanly. Can you wait for the next version and see it is fixed, or can you send me the file to test? christos From dnovotny at redhat.com Mon Oct 13 18:51:35 2008 From: dnovotny at redhat.com (Daniel Novotny) Date: Mon, 13 Oct 2008 11:51:35 -0400 (EDT) Subject: a patch for a crash in a .xlx file In-Reply-To: <933241303.52611223912703274.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <1626181023.54471223913095345.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> hello, since the rpmbuild problem does not bother us anymore, I will wait for the next version thanks and regards, Daniel Novotny ----- "Christos Zoulas" wrote: > On Oct 13, 7:56am, dnovotny at redhat.com (Daniel Novotny) wrote: > -- Subject: a patch for a crash in a .xlx file > > | Hello, > | > | we have run into a problem with one specific file which caused a > "vasprintf failed" crash: > | > | http://people.fedoraproject.org/~dnovotny/miXart8.xlx was > incorrectly classified as a "DOS executable (device driver)", > | because it begins with 0xffffffff and has further characteristics, > according to "magic/Magdir/msdos" magicfile > | > | "file" tries to print the "device driver name" and fails, returning > error to the parent process, making domino effect in rpmbuild(8) > | > | my patch defines the "DOS executable (device driver)" more > precisely, so there is no "string x" > | ("any" string) in the magic, but the specific names of the specific > device drivers instead. this way the crash is averted. > | > | patch attached > > I am not sure that hard-coding the driver names is the besit way of > going > about it and the guy that wrote most of thisd stuff has made a lot of > changes > making your patch not to apply cleanly. Can you wait for the next > version > and see it is fixed, or can you send me the file to test? > > christos > > _______________________________________________ > File mailing list > File at mx.gw.com > http://mx.gw.com/mailman/listinfo/file From dnovotny at redhat.com Thu Oct 16 14:20:48 2008 From: dnovotny at redhat.com (Daniel Novotny) Date: Thu, 16 Oct 2008 07:20:48 -0400 (EDT) Subject: --mime-encoding option problems In-Reply-To: <901938201.1200261224155928860.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <21594783.1200451224156048255.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Hello, we discovered, that the --mime-encoding option does not behave according to RFC2045 MIME encoding definition, when reading files, which are analyzed with softmagic: [lkundrak at trurl ~]$ file --mime-encoding /bin/ls /bin/ls: application/x-executable [lkundrak at trurl ~]$ the patch I attach works like this: [danny at localhost Download]$ file --mime-encoding /etc/passwd /etc/passwd: 7bit [danny at localhost Download]$ file --mime-encoding /bin/ls /bin/ls: binary further research discovered, that the option segfaults (with or without my patch) with empty files, directories and such: [danny at localhost Download]$ file --mime-encoding /etc Segmentation fault I have not look into this yet, since it's another problem: what do you think of this? have a nice day, Daniel Novotny -------------- next part -------------- A non-text attachment was scrubbed... Name: file-4.26-mime-encoding.patch Type: text/x-patch Size: 1300 bytes Desc: not available URL: From christos at zoulas.com Thu Oct 16 19:32:17 2008 From: christos at zoulas.com (Christos Zoulas) Date: Thu, 16 Oct 2008 12:32:17 -0400 Subject: --mime-encoding option problems In-Reply-To: <21594783.1200451224156048255.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> from Daniel Novotny (Oct 16, 7:20am) Message-ID: <20081016163217.E41DB5654F@rebar.astron.com> On Oct 16, 7:20am, dnovotny at redhat.com (Daniel Novotny) wrote: -- Subject: --mime-encoding option problems | Hello, | | we discovered, that the --mime-encoding option does not behave according to RFC2045 MIME encoding definition, when | reading files, which are analyzed with softmagic: | | [lkundrak at trurl ~]$ file --mime-encoding /bin/ls | /bin/ls: application/x-executable | [lkundrak at trurl ~]$ | | the patch I attach works like this: | | [danny at localhost Download]$ file --mime-encoding /etc/passwd | /etc/passwd: 7bit | [danny at localhost Download]$ file --mime-encoding /bin/ls | /bin/ls: binary | | further research discovered, that the option segfaults (with or without my patch) with empty files, directories and such: | | [danny at localhost Download]$ file --mime-encoding /etc | Segmentation fault | | I have not look into this yet, since it's another problem: what do you think of this? | | have a nice day, | | Daniel Novotny You are absolutely right and this is broken (very broken actually). I've already fixed it in my tree and the fixes will be out for the next release. thanks, christos From christos at zoulas.com Fri Oct 17 21:21:12 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 17 Oct 2008 14:21:12 -0400 Subject: CDF support in file Message-ID: <20081017182112.BE4C85654E@rebar.astron.com> I finally got the time and wrote code to parse microsoft document files. This means that now you get: ./file -m ../magic/magic.mgc office/Doc1.doc office/Doc1.doc: CDF V2 Document, Little Endian, Os: Windows, Version 6.0, Code page: 1252, Author: christos zoulas, Template: Normal.dotm, Last Saved By: christos zoulas, Revision Number: 1, Name of Creating Application: Microsoft Office Word, Total Editing Time: 01:00, Create Time/Date: Thu Oct 9 22:56:00 2008, Last Saved Time/Date: Thu Oct 9 22:57:00 2008, Number of Pages: 1, Number of Words: 0, Number of Characters: 0, Security: 0 I also fixed a bunch of issues with --mime-encoding, and due to the number of code changes I don't want to release it to the unsuspecting public before it is tested a bit more. If you have time, please download it from: ftp://ftp.astron.com/pri/file-4.26.1.tar.gz and let me know what you think. Best, christos From amish at net-square.com Wed Oct 22 14:58:05 2008 From: amish at net-square.com (Amish Shah) Date: Wed, 22 Oct 2008 17:28:05 +0530 Subject: 'file' for x64 bit win2003 machines.. Message-ID: <48FF154D.4010407@net-square.com> Greetings Friends, We're planning to use 'file' lib for the filetype detection in our application. The application will also suppose to work with x64 bit machine. I'm wondering, is there a setup or prebuilt libraries available for x64? At the same time, I'm also trying to compile 'file' lib with source, with the use of msys/mingw. Is there any straight procedure available by which I can built the 'file' lib? Has any body tried to compile same on x64 ? Is there a good resources or links available for building up gnuwin32 lib by ourselves ? Thanks & Regards, --Amish From tksmashiw at gmail.com Wed Oct 29 14:55:24 2008 From: tksmashiw at gmail.com (tks) Date: Wed, 29 Oct 2008 21:55:24 +0900 Subject: Xcursor data format Message-ID: <20fa13e40810290555la607bb5v4aea50ed8bfd84ba@mail.gmail.com> Hi! Could you add Xcursor for your file magic list? ======= # Xcursor data # X11 mouse cursor format defined in libXcursor, see # http://www.x.org/archive/X11R6.8.1/doc/Xcursor.3.html # http://cgit.freedesktop.org/xorg/lib/libXcursor/tree/include/X11/Xcursor/Xcursor.h 0 string Xcur Xcursor data !:mime image/x-xcursor >10 leshort x version %hd >>8 leshort x \b.%hd ======= Mime type is not defined anywhere as long as I know, but used by tradition. Thanks for reading. From christos at zoulas.com Wed Oct 29 15:54:25 2008 From: christos at zoulas.com (Christos Zoulas) Date: Wed, 29 Oct 2008 09:54:25 -0400 Subject: Xcursor data format In-Reply-To: <20fa13e40810290555la607bb5v4aea50ed8bfd84ba@mail.gmail.com> from tks (Oct 29, 9:55pm) Message-ID: <20081029135425.F2AB85654E@rebar.astron.com> On Oct 29, 9:55pm, tksmashiw at gmail.com (tks) wrote: -- Subject: Xcursor data format | Hi! | Could you add Xcursor for your file magic list? Added, thanks. christos From dnovotny at redhat.com Wed Nov 12 19:16:05 2008 From: dnovotny at redhat.com (Daniel Novotny) Date: Wed, 12 Nov 2008 12:16:05 -0500 (EST) Subject: magic entry for Altera Nios II binaries In-Reply-To: <93208515.2070271226509970236.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <1966672409.2071501226510165436.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> hello, one of Fedora users added a magic entry for Altera Nios II binaries to ELF executables magic sending this here, so you can add it to the next release... have a nice day, Daniel Novotny -------------- next part -------------- A non-text attachment was scrubbed... Name: file-4.26-alteranios.patch Type: text/x-patch Size: 492 bytes Desc: not available URL: From vapier at gentoo.org Sat Nov 29 01:55:44 2008 From: vapier at gentoo.org (Mike Frysinger) Date: Fri, 28 Nov 2008 18:55:44 -0500 Subject: Fwd: Patch to file-4.26 to allow build of magic file Message-ID: <200811281855.45398.vapier@gentoo.org> ---------- Forwarded Message ---------- Subject: Patch to file-4.26 to allow build of magic file Date: Tuesday 11 November 2008 From: Bill To: vapier at gentoo.org, cardoe at gentoo.org, ranger at gentoo.org Hello, The magic file magic/Magdir/epoc has first line commented out, similar to earlier bug which was reported and patched. Here's the patch to that file to allow file-4.26 to emerge: --- magic/Magdir/epoc.orig 2009-11-11 09:52:08.000000000 -0600 +++ magic/Magdir/epoc 2008-11-11 09:52:11.000000000 -0600 @@ -2,7 +2,7 @@ #------------------------------------------------------------------------------ # Epoc 32 : file(1) magic for Epoc Documents [psion/osaris # Stefan Praszalowicz (hpicollo at worldnet.fr) -#0 lelong 0x10000037 Epoc32 +0 lelong 0x10000037 Epoc32 >4 lelong 0x1000006D >>8 lelong 0x1000007F Word >>8 lelong 0x10000088 Sheet ------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: This is a digitally signed message part. URL: From christos at zoulas.com Sat Nov 29 02:33:43 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 28 Nov 2008 19:33:43 -0500 Subject: Fwd: Patch to file-4.26 to allow build of magic file In-Reply-To: <200811281855.45398.vapier@gentoo.org> from Mike Frysinger (Nov 28, 6:55pm) Message-ID: <20081129003343.3238F5654E@rebar.astron.com> On Nov 28, 6:55pm, vapier at gentoo.org (Mike Frysinger) wrote: -- Subject: Fwd: Patch to file-4.26 to allow build of magic file | Hello, | | The magic file magic/Magdir/epoc has first line commented out, similar to | earlier bug which was reported and patched. Here's the patch to that file | to allow file-4.26 to emerge: I changed it to look like: 0 lelong 0x10000037 >4 lelong 0x1000006D Epoc32 >>8 lelong 0x1000007F Word >>8 lelong 0x10000088 Sheet christos From dnovotny at redhat.com Mon Dec 1 17:36:09 2008 From: dnovotny at redhat.com (Daniel Novotny) Date: Mon, 1 Dec 2008 10:36:09 -0500 (EST) Subject: incorrect recognition: documentation as Perl5 modules In-Reply-To: <131305877.399221228145279789.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <1992114827.404001228145769275.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> hello, the attached file is incorrectly recognized as "Perl5 module source text" - in fact it's a documentation about Perl and not Perl script itself :) This confuses rpmbuild a lot (and causes build problems) ... seems this magic entry # by Dmitry V. Levin and Alexey Tourbin # check the first line 0 search/1 package 0 regex \^package[\ \t]+[A-Za-z_] >0 regex \^package[\ \t]+[0-9A-Za-z_:]+\ *; Perl5 module source text # not 'p', check other lines 0 search/1 !p >0 regex \^package[\ \t]+[0-9A-Za-z_:]+\ *; >>0 regex \^1\ *;|\^(use|sub|my)\ .*[(;{=] Perl5 module source text is the cause... can it be refined somehow? regards, Daniel Novotny -------------- next part -------------- An HTML attachment was scrubbed... URL: From christos at zoulas.com Tue Dec 2 18:26:00 2008 From: christos at zoulas.com (Christos Zoulas) Date: Tue, 2 Dec 2008 11:26:00 -0500 Subject: incorrect recognition: documentation as Perl5 modules In-Reply-To: <1992114827.404001228145769275.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> from Daniel Novotny (Dec 1, 10:36am) Message-ID: <20081202162600.B66A05654E@rebar.astron.com> On Dec 1, 10:36am, dnovotny at redhat.com (Daniel Novotny) wrote: -- Subject: incorrect recognition: documentation as Perl5 modules | hello, | | the attached file is incorrectly recognized as "Perl5 module source text" | - in fact it's a documentation about Perl and not Perl script itself :) | This confuses rpmbuild a lot (and causes build problems) ... | | seems this magic entry | | # by Dmitry V. Levin and Alexey Tourbin | # check the first line | 0 search/1 package | 0 regex \^package[\ \t]+[A-Za-z_] Delete the above line. It is wrong. thanks, christos From andrey.fomichev at gmail.com Wed Dec 10 17:52:03 2008 From: andrey.fomichev at gmail.com (Andrey Fomichev) Date: Wed, 10 Dec 2008 18:52:03 +0300 Subject: Detecting Office 2007 file formats Message-ID: <7df29e8b0812100752k64a0a02evd590fa52704238b6@mail.gmail.com> Hello, *Christos, *The List, Sorry if this question was already asked. I tried to search archive but without success. Does libmagic have support for MS Office 2007 file formats? I tried the last version of file and it determines docx as a zip archive. $ file wordfile.docx wordfile.docx: Zip archive data, at least v2.0 to extract $ file --version file-4.26 Have anyone considered adding support for Office 2007 file formats? It seems that all files begin with 0x50 0x4b 0x03 0x04 etc. Seems pretty simple... But I haven't figured out yet how to distinguish docx from, say, xlsx. Thanks in advance, Andrey Fomichev -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrey.fomichev at gmail.com Wed Dec 10 19:02:11 2008 From: andrey.fomichev at gmail.com (Andrey Fomichev) Date: Wed, 10 Dec 2008 20:02:11 +0300 Subject: Detecting Office 2007 file formats In-Reply-To: <7df29e8b0812100752k64a0a02evd590fa52704238b6@mail.gmail.com> References: <7df29e8b0812100752k64a0a02evd590fa52704238b6@mail.gmail.com> Message-ID: <7df29e8b0812100902m3d23c3a0nf3397dd4eeea28b3@mail.gmail.com> My previous magic numbers 0x50 0x4b 0x03 0x04 were corresponded to application/zip exactly, that seems quite natural taking into account that new MS Office file formats are compressed xml. These magic numbers below looks better. Word 2007 (.docx) application/vnd.openxmlformats-officedocument.wordprocessingml.document String in HEX: 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 30 c9 Excel 2007 (xltx) application/vnd.openxmlformats-officedocument.spreadsheetml.template String in HEX: 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 a7 95 PowerPoint 2007 (pptx) application/vnd.openxmlformats-officedocument.presentationml.presentation String in HEX: 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 5c 0b Best regards, Andrey On Wed, Dec 10, 2008 at 6:52 PM, Andrey Fomichev wrote: > Hello, *Christos, *The List, > > Sorry if this question was already asked. I tried to search archive but > without success. Does libmagic have support for MS Office 2007 file formats? > I tried the last version of file and it determines docx as a zip archive. > > $ file wordfile.docx > wordfile.docx: Zip archive data, at least v2.0 to extract > $ file --version > file-4.26 > > Have anyone considered adding support for Office 2007 file formats? It > seems that all files begin with 0x50 0x4b 0x03 0x04 etc. Seems pretty > simple... But I haven't figured out yet how to distinguish docx from, say, > xlsx. > > Thanks in advance, > Andrey Fomichev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christos at zoulas.com Thu Dec 11 18:32:32 2008 From: christos at zoulas.com (Christos Zoulas) Date: Thu, 11 Dec 2008 11:32:32 -0500 Subject: Detecting Office 2007 file formats In-Reply-To: <7df29e8b0812100752k64a0a02evd590fa52704238b6@mail.gmail.com> from "Andrey Fomichev" (Dec 10, 6:52pm) Message-ID: <20081211163232.69A545654E@rebar.astron.com> On Dec 10, 6:52pm, andrey.fomichev at gmail.com ("Andrey Fomichev") wrote: -- Subject: Detecting Office 2007 file formats | Hello, *Christos, *The List, | | Sorry if this question was already asked. I tried to search archive but | without success. Does libmagic have support for MS Office 2007 file formats? | I tried the last version of file and it determines docx as a zip archive. | | $ file wordfile.docx | wordfile.docx: Zip archive data, at least v2.0 to extract | $ file --version | file-4.26 | | Have anyone considered adding support for Office 2007 file formats? It seems | that all files begin with 0x50 0x4b 0x03 0x04 etc. Seems pretty simple... | But I haven't figured out yet how to distinguish docx from, say, xlsx. | | Thanks in advance, | Andrey Fomichev I have put some thought on it, and the missing part now is a good opensource zip library. Then we can unzip, look inside docProps/app.xml and docProps/core.xml and get the information we need. I hope to have something soon, but not for the next revision of file which already has too many changes. For those who want a preview of the next version that supports CDF files (Office 2003 or less) and ID3 files: ftp://ftp.astron.com:/pri/file-4.26.tar.gz Please report any bugs... christos From christos at zoulas.com Thu Dec 11 18:45:41 2008 From: christos at zoulas.com (Christos Zoulas) Date: Thu, 11 Dec 2008 11:45:41 -0500 Subject: Detecting Office 2007 file formats In-Reply-To: <7df29e8b0812100902m3d23c3a0nf3397dd4eeea28b3@mail.gmail.com> from "Andrey Fomichev" (Dec 10, 8:02pm) Message-ID: <20081211164541.E71285654E@rebar.astron.com> On Dec 10, 8:02pm, andrey.fomichev at gmail.com ("Andrey Fomichev") wrote: -- Subject: Re: Detecting Office 2007 file formats | | --===============0782490190== | Content-Type: multipart/alternative; | boundary="----=_Part_96500_25505578.1228928531366" | | | ------=_Part_96500_25505578.1228928531366 | Content-Type: text/plain; charset=ISO-8859-1 | Content-Transfer-Encoding: 7bit | Content-Disposition: inline | | My previous magic numbers 0x50 0x4b 0x03 0x04 were corresponded to | application/zip exactly, that seems quite natural taking into account that | new MS Office file formats are compressed xml. These magic numbers below | looks better. | | Word 2007 (.docx) | application/vnd.openxmlformats-officedocument.wordprocessingml.document | String in HEX: 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 30 c9 | | Excel 2007 (xltx) | application/vnd.openxmlformats-officedocument.spreadsheetml.template | String in HEX: 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 a7 95 | | PowerPoint 2007 (pptx) | application/vnd.openxmlformats-officedocument.presentationml.presentation | String in HEX: 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 5c 0b | | Best regards, | Andrey That would be a good interim solution. christos From lists-file at cappella.us Thu Dec 11 21:19:34 2008 From: lists-file at cappella.us (Mike Cappella) Date: Thu, 11 Dec 2008 11:19:34 -0800 Subject: Detecting Office 2007 file formats In-Reply-To: <20081211163232.69A545654E@rebar.astron.com> References: <20081211163232.69A545654E@rebar.astron.com> Message-ID: <494167C6.1010605@cappella.us> Christos Zoulas wrote: > > I have put some thought on it, and the missing part now is a good > opensource zip library. Then we can unzip, look inside docProps/app.xml > and docProps/core.xml and get the information we need. I hope to have > something soon, but not for the next revision of file which already > has too many changes. For those who want a preview of the next version > that supports CDF files (Office 2003 or less) and ID3 files: > > ftp://ftp.astron.com:/pri/file-4.26.tar.gz > > Please report any bugs... > > christos I gave a quick test of 3 word documents. ( 4.25 indicated "OLE 2 Compound Document" for all, and 4.26 (public) indicated "Microsoft Office Document Microsoft Word Document". ) A simple Word 10 document: Word10.doc: CDF V2 Document, Little Endian, Os: Windows, Version 5.1, Code page: 1252, Title: 1, Author: Mike Cappella, Template: Normal.dot, Last Saved By: Mike Cappella, Revision Number: 1, Name of Creating Application: Microsoft Word 10.0, Total Editing Time: 01:00, Create Time/Date: Tue Jul 16 20:11:00 2002, Last Saved Time/Date: Tue Jul 16 20:12:00 2002, Number of Pages: 1, Number of Words: 18, Number of Characters: 110, Security: 0 A simple Word 11 document: Word11 simple.doc: CDF V2 Document, Little Endian, Os: Windows, Version 5.1, Code page: 1252, Title: , Subject: , Author: Mike Cappella, Keywords: , Template: Normal.dot, Last Saved By: Mike Cappella, Revision Number: 2, Name of Creating Application: Microsoft Office Word, Create Time/Date: Wed Dec 10 18:19:00 2008, Last Saved Time/Date: Wed Dec 10 18:19:00 2008, Number of Pages: 1, Number of Words: 1, Number of Characters: 7, Security: 0 A complex Word 11 document (w/embedded excel worksheet): Word11 complex.doc: CDF V2 Document, Little Endian, Os: Windows, Version 5.1, Code page: 1252, Author: Mike Cappella, Last Saved By: Mike Cappella, Name of Creating Application: Microsoft Excel, Create Time/Date: Wed Dec 10 18:17:45 2008, Last Saved Time/Date: Wed Dec 10 18:17:48 2008, Security: 0 It indicates the creating application is Excel, but it was Word 11. I don't know the CDF spec, so perhaps this is the best data available, or perhaps its a bug? Mike From Mark.Martinec+amavis at ijs.si Thu Dec 11 23:18:50 2008 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Thu, 11 Dec 2008 22:18:50 +0100 Subject: Detecting Office 2007 file formats Message-ID: <200812112218.50554.Mark.Martinec+amavis@ijs.si> > > For those who want a preview of the next version > > that supports CDF files (Office 2003 or less) and ID3 files: > > ftp://ftp.astron.com:/pri/file-4.26.tar.gz > > Please report any bugs... Running that version of file(1), I got two messages stuck in a mail queue today, each attempt of content checking fails with a timeout during a call to file(1). It turns out they contain an 8 MB doc file. Running file(1) manually on that document has been spinning CPU for the last two hours, and doesn't seem to have intention of finishing. If the new code cannot be contained within reasonable time bounds, there should be a way to disable it (and perhaps disabled by default). Mark From ian at darwinsys.com Thu Dec 11 23:25:26 2008 From: ian at darwinsys.com (Ian Darwin) Date: Thu, 11 Dec 2008 16:25:26 -0500 Subject: MS-Office file formats In-Reply-To: <494167C6.1010605@cappella.us> References: <20081211163232.69A545654E@rebar.astron.com> <494167C6.1010605@cappella.us> Message-ID: <49418546.6030209@darwinsys.com> > A simple Word 10 document: > > Word10.doc: CDF V2 Document, Little Endian, Os: Windows, > Version 5.1, Code page: 1252, Title: 1, Author: Mike Cappella, Template: > Normal.dot, Last Saved By: Mike Cappella, Revision Number: 1, Name of > Creating Application: Microsoft Word 10.0, Total Editing Time: 01:00, > Create Time/Date: Tue Jul 16 20:11:00 2002, Last Saved Time/Date: Tue > Jul 16 20:12:00 2002, Number of Pages: 1, Number of Words: 18, Number of > Characters: 110, Security: 0 > You call that simple? :-) Geez, do we need to print every imaginable field from M$-Word docs? Is the time it took to edit the last time somebody changed a letter really to be considered part of the *file type*? Just because we can print some factoid doesn't mean we should :-) If users really are demanding this, there should be a --extra-tags or --all-info or even --mega-bloated-output :-) option that would do this uniformly for e.g., TIFF file format, anything that saves a huge amound of extra information. It should not be the default. One line (80-100 chars max) should be enough to convey the file format. Thanks for listening. Ian From ian at darwinsys.com Thu Dec 11 23:26:43 2008 From: ian at darwinsys.com (Ian Darwin) Date: Thu, 11 Dec 2008 16:26:43 -0500 Subject: Detecting Office 2007 file formats In-Reply-To: <200812112218.50554.Mark.Martinec+amavis@ijs.si> References: <200812112218.50554.Mark.Martinec+amavis@ijs.si> Message-ID: <49418593.2050407@darwinsys.com> Mark Martinec wrote: >>> For those who want a preview of the next version >>> that supports CDF files (Office 2003 or less) and ID3 files: >>> ftp://ftp.astron.com:/pri/file-4.26.tar.gz >>> Please report any bugs... >>> > > Running that version of file(1), I got two messages stuck in a > mail queue today, each attempt of content checking fails > with a timeout during a call to file(1). > > It turns out they contain an 8 MB doc file. Running file(1) manually > on that document has been spinning CPU for the last two hours, > and doesn't seem to have intention of finishing. > > If the new code cannot be contained within reasonable time bounds, > there should be a way to disable it (and perhaps disabled by default). > I guess there's a pragmatic argument supporting my prosaic argument. From christos at zoulas.com Fri Dec 12 00:32:51 2008 From: christos at zoulas.com (Christos Zoulas) Date: Thu, 11 Dec 2008 17:32:51 -0500 Subject: Detecting Office 2007 file formats In-Reply-To: <494167C6.1010605@cappella.us> from Mike Cappella (Dec 11, 11:19am) Message-ID: <20081211223251.9526D5654E@rebar.astron.com> On Dec 11, 11:19am, lists-file at cappella.us (Mike Cappella) wrote: -- Subject: Re: Detecting Office 2007 file formats | Christos Zoulas wrote: | > | > I have put some thought on it, and the missing part now is a good | > opensource zip library. Then we can unzip, look inside docProps/app.xml | > and docProps/core.xml and get the information we need. I hope to have | > something soon, but not for the next revision of file which already | > has too many changes. For those who want a preview of the next version | > that supports CDF files (Office 2003 or less) and ID3 files: | > | > ftp://ftp.astron.com:/pri/file-4.26.tar.gz | > | > Please report any bugs... | > | > christos | | I gave a quick test of 3 word documents. ( 4.25 indicated "OLE 2 | Compound Document" for all, and 4.26 (public) indicated "Microsoft | Office Document Microsoft Word Document". ) | | A simple Word 10 document: | | Word10.doc: CDF V2 Document, Little Endian, Os: Windows, | Version 5.1, Code page: 1252, Title: 1, Author: Mike Cappella, Template: | Normal.dot, Last Saved By: Mike Cappella, Revision Number: 1, Name of | Creating Application: Microsoft Word 10.0, Total Editing Time: 01:00, | Create Time/Date: Tue Jul 16 20:11:00 2002, Last Saved Time/Date: Tue | Jul 16 20:12:00 2002, Number of Pages: 1, Number of Words: 18, Number of | Characters: 110, Security: 0 | | A simple Word 11 document: | | Word11 simple.doc: CDF V2 Document, Little Endian, Os: Windows, | Version 5.1, Code page: 1252, Title: , Subject: , Author: Mike Cappella, | Keywords: , Template: Normal.dot, Last Saved By: Mike Cappella, Revision | Number: 2, Name of Creating Application: Microsoft Office Word, Create | Time/Date: Wed Dec 10 18:19:00 2008, Last Saved Time/Date: Wed Dec 10 | 18:19:00 2008, Number of Pages: 1, Number of Words: 1, Number of | Characters: 7, Security: 0 | | A complex Word 11 document (w/embedded excel worksheet): | | Word11 complex.doc: CDF V2 Document, Little Endian, Os: Windows, | Version 5.1, Code page: 1252, Author: Mike Cappella, Last Saved By: Mike | Cappella, Name of Creating Application: Microsoft Excel, Create | Time/Date: Wed Dec 10 18:17:45 2008, Last Saved Time/Date: Wed Dec 10 | 18:17:48 2008, Security: 0 | | It indicates the creating application is Excel, but it was Word 11. I | don't know the CDF spec, so perhaps this is the best data available, or | perhaps its a bug? It is probably a bug. The url to the spec is documented in the sources. CDF looks like a filesystem (FAT). CDF files have sectors and directories. Currently I look at the first directory entry of a certain type, perhaps I need to look at the leaf entry or the root entry of that? You can cc -g -c -DTEST -DDEBUG cdf.c cc -g -c -DDEBUG cdf_time.c cc -g cdf.o cdf_time.o -o cdf ./cdf Word11\ complex.doc and see a full dump... christos From christos at zoulas.com Fri Dec 12 00:36:27 2008 From: christos at zoulas.com (Christos Zoulas) Date: Thu, 11 Dec 2008 17:36:27 -0500 Subject: Detecting Office 2007 file formats In-Reply-To: <200812112218.50554.Mark.Martinec+amavis@ijs.si> from Mark Martinec (Dec 11, 10:18pm) Message-ID: <20081211223627.61CF25654E@rebar.astron.com> On Dec 11, 10:18pm, Mark.Martinec+amavis at ijs.si (Mark Martinec) wrote: -- Subject: Re: Detecting Office 2007 file formats | > > For those who want a preview of the next version | > > that supports CDF files (Office 2003 or less) and ID3 files: | > > ftp://ftp.astron.com:/pri/file-4.26.tar.gz | > > Please report any bugs... | | Running that version of file(1), I got two messages stuck in a | mail queue today, each attempt of content checking fails | with a timeout during a call to file(1). | | It turns out they contain an 8 MB doc file. Running file(1) manually | on that document has been spinning CPU for the last two hours, | and doesn't seem to have intention of finishing. | | If the new code cannot be contained within reasonable time bounds, | there should be a way to disable it (and perhaps disabled by default). --exclude cdf That is a bug in the code and this is why it is not released yet :-) If you can send me the file, I can try to fix it... christos From christos at zoulas.com Fri Dec 12 00:39:20 2008 From: christos at zoulas.com (Christos Zoulas) Date: Thu, 11 Dec 2008 17:39:20 -0500 Subject: MS-Office file formats In-Reply-To: <49418546.6030209@darwinsys.com> from Ian Darwin (Dec 11, 4:25pm) Message-ID: <20081211223920.569765654E@rebar.astron.com> On Dec 11, 4:25pm, ian at darwinsys.com (Ian Darwin) wrote: -- Subject: MS-Office file formats | | > A simple Word 10 document: | > | > Word10.doc: CDF V2 Document, Little Endian, Os: Windows, | > Version 5.1, Code page: 1252, Title: 1, Author: Mike Cappella, Template: | > Normal.dot, Last Saved By: Mike Cappella, Revision Number: 1, Name of | > Creating Application: Microsoft Word 10.0, Total Editing Time: 01:00, | > Create Time/Date: Tue Jul 16 20:11:00 2002, Last Saved Time/Date: Tue | > Jul 16 20:12:00 2002, Number of Pages: 1, Number of Words: 18, Number of | > Characters: 110, Security: 0 | > | You call that simple? :-) | | Geez, do we need to print every imaginable field from M$-Word docs? Is | the time it took to edit the last time somebody changed a letter really | to be considered part of the *file type*? Just because we can print some | factoid doesn't mean we should :-) | | If users really are demanding this, there should be a --extra-tags or | --all-info or even --mega-bloated-output :-) option that would do this | uniformly for e.g., TIFF file format, anything that saves a huge amound | of extra information. It should not be the default. | One line (80-100 chars max) should be enough to convey the file format. | | Thanks for listening. | Ian That is a good idea. I need to figure out how to trim entries in the magic file. One way is to have verbose or non-verbose flags for each magic line entry. christos From andrey.fomichev at gmail.com Fri Dec 12 13:16:13 2008 From: andrey.fomichev at gmail.com (Andrey Fomichev) Date: Fri, 12 Dec 2008 14:16:13 +0300 Subject: Detecting Office 2007 file formats In-Reply-To: <20081211164541.E71285654E@rebar.astron.com> References: <7df29e8b0812100902m3d23c3a0nf3397dd4eeea28b3@mail.gmail.com> <20081211164541.E71285654E@rebar.astron.com> Message-ID: <7df29e8b0812120316tf7e70a7w2dbce95671286882@mail.gmail.com> On Thu, Dec 11, 2008 at 7:45 PM, Christos Zoulas wrote: > On Dec 10, 8:02pm, andrey.fomichev at gmail.com ("Andrey Fomichev") wrote: > -- Subject: Re: Detecting Office 2007 file formats > > | > | --===============0782490190== > | Content-Type: multipart/alternative; > | boundary="----=_Part_96500_25505578.1228928531366" > | > | > | ------=_Part_96500_25505578.1228928531366 > | Content-Type: text/plain; charset=ISO-8859-1 > | Content-Transfer-Encoding: 7bit > | Content-Disposition: inline > | > | My previous magic numbers 0x50 0x4b 0x03 0x04 were corresponded to > | application/zip exactly, that seems quite natural taking into account > that > | new MS Office file formats are compressed xml. These magic numbers below > | looks better. > | > | Word 2007 (.docx) > | application/vnd.openxmlformats-officedocument.wordprocessingml.document > | String in HEX: 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 30 c9 > | > | Excel 2007 (xltx) > | application/vnd.openxmlformats-officedocument.spreadsheetml.template > | String in HEX: 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 a7 95 > | > | PowerPoint 2007 (pptx) > | application/vnd.openxmlformats-officedocument.presentationml.presentation > | String in HEX: 50 4b 03 04 14 00 06 00 08 00 00 00 21 00 5c 0b > | > | Best regards, > | Andrey > > That would be a good interim solution. Just for the history. The magic numbers above are incorrect. I found docx documents with signature that differ from one above. According to Open XML spec bytes 30 c9 are part of zip header (?) checksum. It seems that there is a good way to look at a string starting from byte 31. If the string is equal to "[Content_Types].xml" than you have Open XML archive (don't forget to check 4 first bytes 50 4b 03 04 ). To determine the exact mime type we should look into the archive. The next rule could be "try to find a word 'word' in next N bytes". In case of success you have vnd.openxmlformats-officedocument.wordprocessingml.document. But it's not a very good recipe... Regards, Andrey > > > christos > > _______________________________________________ > File mailing list > File at mx.gw.com > http://mx.gw.com/mailman/listinfo/file > -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrey.fomichev at gmail.com Fri Dec 12 14:22:02 2008 From: andrey.fomichev at gmail.com (Andrey Fomichev) Date: Fri, 12 Dec 2008 15:22:02 +0300 Subject: Performance difference between file-4.23 and file-4.26 Message-ID: <7df29e8b0812120422q501aee15lef42352610b706ac@mail.gmail.com> Hello! I made a simple test to evaluate performance of libmagic. I might be doing something wrong, but I found significant performance difference between versions 4.23 (the default version in my FreeBSD 7 installation) and 4.26 downloaded from the official site. 100000 checks ~ 1 sec 100000 checks ~ 40 sec The test were quite simple. Before test load first 1KB of a file (totally 8 files) into memory. In cycle call magic_buffer function on every buf. I can provide sources if required. It's appeared that 4.23 is 40 times faster than 4.26. I tried to throw away some files form Magic dir, but performance improved only a little bit. I recompiled magic file with only the following sources animation flash jpeg msdos rtf audio images mail.news pdf sgml Note: I tested libmagic on 8 files, they are: html, doc, cpp, pdf, docx, xslx, pptx. Does somebody monitor the performance of libmagic? How could you explain such significant performance difference? Just curious, what is the architecture of libmagic? Do you check every rule against input file or buffer? Or do you use a finite state machine? Regards, Andrey -------------- next part -------------- An HTML attachment was scrubbed... URL: From christos at zoulas.com Fri Dec 12 16:09:53 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 12 Dec 2008 09:09:53 -0500 Subject: Performance difference between file-4.23 and file-4.26 In-Reply-To: <7df29e8b0812120422q501aee15lef42352610b706ac@mail.gmail.com> from "Andrey Fomichev" (Dec 12, 3:22pm) Message-ID: <20081212140953.153EF5654E@rebar.astron.com> On Dec 12, 3:22pm, andrey.fomichev at gmail.com ("Andrey Fomichev") wrote: -- Subject: Performance difference between file-4.23 and file-4.26 | Hello! | | I made a simple test to evaluate performance of libmagic. I might be doing | something wrong, but I found significant performance difference between | versions 4.23 (the default version in my FreeBSD 7 installation) and 4.26 | downloaded from the official site. | | 100000 checks ~ 1 sec | 100000 checks ~ 40 sec | | The test were quite simple. Before test load first 1KB of a file (totally 8 | files) into memory. In cycle call magic_buffer function on every buf. I can | provide sources if required. | It's appeared that 4.23 is 40 times faster than 4.26. I tried to throw away | some files form Magic dir, but performance improved only a little bit. I | recompiled magic file with only the following sources | animation flash jpeg msdos rtf | audio images mail.news pdf sgml | | Note: I tested libmagic on 8 files, they are: html, doc, cpp, pdf, docx, | xslx, pptx. | | Does somebody monitor the performance of libmagic? How could you explain | such significant performance difference? | | Just curious, what is the architecture of libmagic? Do you check every rule | against input file or buffer? Or do you use a finite state machine? | magic entries are sorted during compilation and then sequentially checked against the given buffer. The first match found is returned unless -k is given. A 40x slowdown is a serious regression, and you should be able to find out what causes it with profiling. christos From andrey.fomichev at gmail.com Fri Dec 12 17:48:56 2008 From: andrey.fomichev at gmail.com (Andrey Fomichev) Date: Fri, 12 Dec 2008 18:48:56 +0300 Subject: Performance difference between file-4.23 and file-4.26 In-Reply-To: <20081212140953.153EF5654E@rebar.astron.com> References: <7df29e8b0812120422q501aee15lef42352610b706ac@mail.gmail.com> <20081212140953.153EF5654E@rebar.astron.com> Message-ID: <7df29e8b0812120748g7510986axd7a804de38f94a85@mail.gmail.com> The problem is in the following lines from magic/Magdir/msdos: # OS/2 batch files are REXX. the second regex is a bit generic, oh well # the matched commands seem to be common in REXX and uncommon elsewhere 100 regex/c =^[\ \t]{0,10}call[\ \t]{1,10}rxfunc OS/2 REXX batch file text 100 regex/c =^[\ \t]{0,10}say\ ['"] OS/2 REXX batch file text regexes are _very_ slow... The result is 100000 checks ~ 2 sec That is two times slower comparing to the version 4.23, which is not good but satisfactory. Have you considered of using FSM generator like Ragel? It should be faster. I understand it's a large and monotonic work... but still. Theoretically, somebody could develop a converter from magic file to Ragel notation. Another question. Since I only need to determine mime type of incoming documents I tried to remove all lines except of those responsible for mime type determination. Hope it will work faster. Have you considered of adding a magic file compilation option to ignore all tests except of mime tests? Regards, Andrey On Fri, Dec 12, 2008 at 5:09 PM, Christos Zoulas wrote: > On Dec 12, 3:22pm, andrey.fomichev at gmail.com ("Andrey Fomichev") wrote: > -- Subject: Performance difference between file-4.23 and file-4.26 > > | Hello! > | > | I made a simple test to evaluate performance of libmagic. I might be > doing > | something wrong, but I found significant performance difference between > | versions 4.23 (the default version in my FreeBSD 7 installation) and 4.26 > | downloaded from the official site. > | > | 100000 checks ~ 1 sec > | 100000 checks ~ 40 sec > | > | The test were quite simple. Before test load first 1KB of a file > (totally 8 > | files) into memory. In cycle call magic_buffer function on every buf. I > can > | provide sources if required. > | It's appeared that 4.23 is 40 times faster than 4.26. I tried to throw > away > | some files form Magic dir, but performance improved only a little bit. I > | recompiled magic file with only the following sources > | animation flash jpeg msdos rtf > | audio images mail.news pdf sgml > | > | Note: I tested libmagic on 8 files, they are: html, doc, cpp, pdf, docx, > | xslx, pptx. > | > | Does somebody monitor the performance of libmagic? How could you explain > | such significant performance difference? > | > | Just curious, what is the architecture of libmagic? Do you check every > rule > | against input file or buffer? Or do you use a finite state machine? > | > > magic entries are sorted during compilation and then sequentially checked > against the given buffer. The first match found is returned unless -k is > given. A 40x slowdown is a serious regression, and you should be able > to find out what causes it with profiling. > > christos > > _______________________________________________ > File mailing list > File at mx.gw.com > http://mx.gw.com/mailman/listinfo/file > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christos at zoulas.com Fri Dec 12 23:01:58 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 12 Dec 2008 16:01:58 -0500 Subject: Detecting Office 2007 file formats In-Reply-To: <20081211223627.61CF25654E@rebar.astron.com> from Christos Zoulas (Dec 11, 5:36pm) Message-ID: <20081212210158.9DDD95654E@rebar.astron.com> On Dec 11, 5:36pm, christos at zoulas.com (Christos Zoulas) wrote: -- Subject: Re: Detecting Office 2007 file formats | On Dec 11, 10:18pm, Mark.Martinec+amavis at ijs.si (Mark Martinec) wrote: | -- Subject: Re: Detecting Office 2007 file formats | | | > > For those who want a preview of the next version | | > > that supports CDF files (Office 2003 or less) and ID3 files: | | > > ftp://ftp.astron.com:/pri/file-4.26.tar.gz | | > > Please report any bugs... | | | | Running that version of file(1), I got two messages stuck in a | | mail queue today, each attempt of content checking fails | | with a timeout during a call to file(1). | | | | It turns out they contain an 8 MB doc file. Running file(1) manually | | on that document has been spinning CPU for the last two hours, | | and doesn't seem to have intention of finishing. | | | | If the new code cannot be contained within reasonable time bounds, | | there should be a way to disable it (and perhaps disabled by default). | | --exclude cdf | | That is a bug in the code and this is why it is not released yet :-) | If you can send me the file, I can try to fix it... The bug is fixed, and there is a new tar file in the same place. christos From daniel.leidert.spam at gmx.net Sat Dec 13 00:01:26 2008 From: daniel.leidert.spam at gmx.net (Daniel Leidert) Date: Fri, 12 Dec 2008 23:01:26 +0100 Subject: snippet-like magic database extension? Message-ID: <1229119286.16087.34.camel@localhost> Hi, Currently the file magic database can be extended by the user via /etc/magic or related files in his home directory. However, this currently means, that an extension to the database can only be made by the user manually (and locally) or by sending you a request to add some format. But there are a lot of applications, which ship their own format - unregistered and unofficial. It would be nice, if such projects could add a snippet to your database during the installation of the project (adding every application/x- officially to file, is IMO not the way to go here). I wonder, if you would think about parsing all files found in a directory, e.g. /etc/magic.d/ or something similar. So people can put their snippets into this directory. But maybe you have reasons to not allow this? I ask this, because I create magic pattern for chemistry related file formats in another project and I have no way to install these pattern to the users system. Comments are appreciated. Regards, Daniel From christos at zoulas.com Sat Dec 13 02:25:42 2008 From: christos at zoulas.com (Christos Zoulas) Date: Fri, 12 Dec 2008 19:25:42 -0500 Subject: snippet-like magic database extension? In-Reply-To: <1229119286.16087.34.camel@localhost> from Daniel Leidert (Dec 12, 11:01pm) Message-ID: <20081213002542.8D69E5654E@rebar.astron.com> On Dec 12, 11:01pm, daniel.leidert.spam at gmx.net (Daniel Leidert) wrote: -- Subject: snippet-like magic database extension? | Hi, | | Currently the file magic database can be extended by the user | via /etc/magic or related files in his home directory. However, this | currently means, that an extension to the database can only be made by | the user manually (and locally) or by sending you a request to add some | format. | | But there are a lot of applications, which ship their own format - | unregistered and unofficial. It would be nice, if such projects could | add a snippet to your database during the installation of the project | (adding every application/x- officially to file, is IMO not the way to | go here). I wonder, if you would think about parsing all files found in | a directory, e.g. /etc/magic.d/ or something similar. So people can put | their snippets into this directory. | | But maybe you have reasons to not allow this? I ask this, because I | create magic pattern for chemistry related file formats in another | project and I have no way to install these pattern to the users system. | It will read a directory of snippets, and also obey $MAGIC to look for the locsation for it. In fact, this is how we build the monolithic file. christos From Mark.Martinec+amavis at ijs.si Sat Dec 13 02:46:06 2008 From: Mark.Martinec+amavis at ijs.si (Mark Martinec) Date: Sat, 13 Dec 2008 01:46:06 +0100 Subject: Detecting Office 2007 file formats In-Reply-To: <20081212210158.9DDD95654E@rebar.astron.com> References: <20081212210158.9DDD95654E@rebar.astron.com> Message-ID: <200812130146.07091.Mark.Martinec+amavis@ijs.si> Christos, > | | It turns out they contain an 8 MB doc file. Running file(1) manually > | | on that document has been spinning CPU for the last two hours, > | | and doesn't seem to have intention of finishing. > The bug is fixed, and there is a new tar file in the same place. Perfect! I'm running this new code again on our mailer, looks good. I'll let you know if some surprise pops up. Thanks Mark From daniel.leidert.spam at gmx.net Sat Dec 13 15:49:49 2008 From: daniel.leidert.spam at gmx.net (Daniel Leidert) Date: Sat, 13 Dec 2008 14:49:49 +0100 Subject: snippet-like magic database extension? In-Reply-To: <20081213002542.8D69E5654E@rebar.astron.com> References: <20081213002542.8D69E5654E@rebar.astron.com> Message-ID: <1229176189.7458.12.camel@localhost> Am Freitag, den 12.12.2008, 19:25 -0500 schrieb Christos Zoulas: > On Dec 12, 11:01pm, daniel.leidert.spam at gmx.net (Daniel Leidert) wrote: > | Currently the file magic database can be extended by the user > | via /etc/magic or related files in his home directory. However, this > | currently means, that an extension to the database can only be made by > | the user manually (and locally) or by sending you a request to add some > | format. > | > | But there are a lot of applications, which ship their own format - > | unregistered and unofficial. It would be nice, if such projects could > | add a snippet to your database during the installation of the project > | (adding every application/x- officially to file, is IMO not the way to > | go here). I wonder, if you would think about parsing all files found in > | a directory, e.g. /etc/magic.d/ or something similar. So people can put > | their snippets into this directory. > | > | But maybe you have reasons to not allow this? I ask this, because I > | create magic pattern for chemistry related file formats in another > | project and I have no way to install these pattern to the users system. > | > > It will read a directory of snippets, and also obey $MAGIC to look > for the locsation for it. Which directory does it read during runtime? The manual pages just tell about the files ~/.magic(.mgc), /etc/magic and the installed /usr/share/file/magic*. With MAGIC one can choose a file. But I don't see a possibility to install a snippet on the users system, which is automatically considered by the file command. > In fact, this is how we build the monolithic file. You speak about build-time (like README)? Unfortunately, this isn't, what I was asking for. I mean, a third party project currently cannot extend the database by installing a snippet to the system. You parse only the above mentioned files during runtime. So I thought, it might be nice to have file further parse a directory during runtime - maybe /etc/magic.d/, where third party projects can install their own snippets to extend the file (magic) database. Regards, Daniel From christos at zoulas.com Sat Dec 13 21:33:21 2008 From: christos at zoulas.com (Christos Zoulas) Date: Sat, 13 Dec 2008 14:33:21 -0500 Subject: snippet-like magic database extension? In-Reply-To: <1229176189.7458.12.camel@localhost> from Daniel Leidert (Dec 13, 2:49pm) Message-ID: <20081213193321.AD48C5654E@rebar.astron.com> On Dec 13, 2:49pm, daniel.leidert.spam at gmx.net (Daniel Leidert) wrote: -- Subject: Re: snippet-like magic database extension? | Am Freitag, den 12.12.2008, 19:25 -0500 schrieb Christos Zoulas: | > On Dec 12, 11:01pm, daniel.leidert.spam at gmx.net (Daniel Leidert) wrote: | | > | Currently the file magic database can be extended by the user | > | via /etc/magic or related files in his home directory. However, this | > | currently means, that an extension to the database can only be made by | > | the user manually (and locally) or by sending you a request to add some | > | format. | > | | > | But there are a lot of applications, which ship their own format - | > | unregistered and unofficial. It would be nice, if such projects could | > | add a snippet to your database during the installation of the project | > | (adding every application/x- officially to file, is IMO not the way to | > | go here). I wonder, if you would think about parsing all files found in | > | a directory, e.g. /etc/magic.d/ or something similar. So people can put | > | their snippets into this directory. | > | | > | But maybe you have reasons to not allow this? I ask this, because I | > | create magic pattern for chemistry related file formats in another | > | project and I have no way to install these pattern to the users system. | > | | > | > It will read a directory of snippets, and also obey $MAGIC to look | > for the locsation for it. | | Which directory does it read during runtime? The manual pages just tell | about the files ~/.magic(.mgc), /etc/magic and the | installed /usr/share/file/magic*. With MAGIC one can choose a file. But | I don't see a possibility to install a snippet on the users system, | which is automatically considered by the file command. | | > In fact, this is how we build the monolithic file. | | You speak about build-time (like README)? Unfortunately, this isn't, | what I was asking for. I mean, a third party project currently cannot | extend the database by installing a snippet to the system. You parse | only the above mentioned files during runtime. | | So I thought, it might be nice to have file further parse a directory | during runtime - maybe /etc/magic.d/, where third party projects can | install their own snippets to extend the file (magic) database. In file-4.26 you can setenv MAGIC /etc/magic.d and file will search there. But compare the speed of parsing the magic files, with the speeed of loading the compiled file. christos From dnovotny at redhat.com Mon Dec 15 18:16:50 2008 From: dnovotny at redhat.com (Daniel Novotny) Date: Mon, 15 Dec 2008 11:16:50 -0500 (EST) Subject: graphviz vs. LaTex: another regexp specified too wide In-Reply-To: <563827400.212991229357544399.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> Message-ID: <1516749057.215171229357810867.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> hello, we stumbled upon a bug in regex: if you use a LaTex \bibliography{} command, file(1) will think it's a graphviz file due to 0 regex/100 [\r\n\t\ ]*graph[\r\n\t\ ]*.*\\{ graphviz graph text the patch I attach changes "*" to "+", because we need the whole word "graph" or "digraph" and so there should be "one or more" whitespace after the keyword... regards, Daniel Novotny -------------- next part -------------- A non-text attachment was scrubbed... Name: file-4.26-graphviz-latex.patch Type: text/x-patch Size: 709 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: test_latex.tex Type: application/x-tex Size: 334 bytes Desc: not available URL: From christos at zoulas.com Mon Dec 15 18:58:35 2008 From: christos at zoulas.com (Christos Zoulas) Date: Mon, 15 Dec 2008 11:58:35 -0500 Subject: graphviz vs. LaTex: another regexp specified too wide In-Reply-To: <1516749057.215171229357810867.JavaMail.root@zmail02.collab.prod.int.phx2.redhat.com> from Daniel Novotny (Dec 15, 11:16am) Message-ID: <20081215165835.226D95654E@rebar.astron.com> On Dec 15, 11:16am, dnovotny at redhat.com (Daniel Novotny) wrote: -- Subject: graphviz vs. LaTex: another regexp specified too wide | hello, | | we stumbled upon a bug in regex: | | if you use a LaTex \bibliography{} command, file(1) will think it's a graphviz file due to | | 0 regex/100 [\r\n\t\ ]*graph[\r\n\t\ ]*.*\\{ graphviz graph text | | the patch I attach changes "*" to "+", because we need the whole word "graph" or "digraph" and | so there should be "one or more" whitespace after the keyword... | | regards, | | Daniel Novotny Ok, I changed it back. Let's see if there are any more complaints. christos