Related
I created a pipeline in data factory and I want to retrieve data from a source for the current month and for the previous month. When I run the pipeline I give the needed parameter named ExtractDate. The format is MM/DD/YYYY .
For the current month I used the following expression in 'Set Variable' activity:
#replace(item().Query,'EXTRACTDATE',formatDateTime(variables('ExtractDate'), 'yyyyMM'))
And for the previous month I tried:
#adddays(variables('ExtractDate'),-28)
The problem appears when the user will set when running the pipeline the date 07/31/2019 for example. Then the previous month will still be July. And if I increase the number to 31, then there is a possibility that the user will introduce 03/01/2019 and from March it will skip the month of February.
I tried to think of a solution, but unfortunately there is no 'addmonths' available in data factory.
Any ideas please?...I've spent 2 days on this issue..
addMonths and addYears are not supported by ADF so far.Please vote up this thread to push the progress.
My trick is use combination of bulit-in functions in ADF. Please see my test:
This month is very simple:
#concat(substring('07/16/2019',6,4),substring('07/16/2019',0,2))
output:
Last month is little complex.It should check if it is the first month of the year.
#if(equals(substring('07/16/2019',0,2),'01'),
concat(
string(sub(
int(substring('07/16/2019',6,4)),1)),
'12'),
string(sub(
int(concat(substring('07/16/2019',6,4),
substring('07/16/2019',0,2))),1)
)
)
if the input param is 01/16/2019,then the output looks like:
My test is based on the static value,please replace it with your variable.
Just for summarize:
The final working dynamic content should be like as below:
#if( equals(variables('SubstringMonth'),'01'),
concat(string(sub(int(variables('SubstringYear')),1)),'12'),
concat(variables('SubstringYear'),string(if(or(equals(sub(int(variables('SubstringMonth')),1),11),equals(sub(int(variables('SubstringMonth')),1),10)),
sub(int(variables('SubstringMonth')),1) ,
concat('0',string(sub(int(variables('SubstringMonth')),1) )))) ))
For the previous month from today you could use
formatDateTime(AddToTime(utcnow(), -1, 'Month'), 'yyyy-MM-dd')
How can I parse the start time output of the ps command? It can parse if it's in the form, like 06:38PM, but when it looks like Tue01PM it somehow can't. I also tried to add cultureinfo, like Convert.ToDateTime(result[8], CultureInfo.GetCultureInfo("en-US")), and with ParseExact like DateTime.ParseExact(result[8], "dddhhhh", CultureInfo.GetCultureInfo("en-US")).
The ps command can be asked to present specific data, and in some cases in specific formats, which can make it easier to parse the results. You request specific data with the -o command line option, whose argument is a comma-separated list of items in the form: item=HEADER. HEADER is the column header to use; leaving it out means that no header will be printed for that column (but if you also leave out the =, you'll get a default header).
There are at least two common items for start time: start and lstart. These give you the standard and long versions of the time, respectively:
$ ps -ostart= 2875
11:28:13
$ ps -olstart= 2875
Wed Mar 18 11:28:13 2015
On Linux, ps gets its information from the "file" /proc/pid/stat, which is a single line containing space-separated fields (see man 5 proc). The start time is the 22nd item in the list; it is an integer, which represents the number of clock ticks since the system was booted. In bash, you can convert this to something more usable with a bit of work:
start_time() {
local now running ticks uptime
now=$(date +%s) # time in seconds since epoch
running=$(cut -d' ' -f22 /proc/$1/stat)
# start time of process in ticks since boot
ticks=$(getconf CLK_TCK) # ticks per second (usually 100)
uptime=($(</proc/uptime)) # seconds since boot and idle time
bc <<<"$now-${uptime[0]}+$running/$ticks"
}
$ date +"%Y-%m-%d %H:%M:%S" -d#$(start_time 2875)
2015-03-18 11:28:12
In linux, you can override the system date format and give it whatever you'd like. Then you can be sure of the output. That is what I would do.
Gives the starting time in epoc of pid / process $MYPID
MYPID=$PPID
STARTING_EPOC_OF_MYPID=$(date +%s -d "$(ps -olstart= ${MYPID})")
echo $STARTING_EPOC_OF_MYPID
I'm not quite sure, if this differs from above.
I am trying to develop a scheduler- and calendar-dependent event application in C#, for which a crucial requirement is to represent recurring events in the database.
What is the best way to represent recurring events in a database?
More Details:
While creating the event I am also sending invites to the certain users and the invitees should be allowed to login to the meeting only during the specified window(meeting duration) or may be decline the login when the invitee attempts to login say, 5 minutes before the scheduled start of the meeting.
The sysjobs, sysjobsschedule and sysschedules tables in SQL Server does a pretty good job of this. I wouldn't reinvent the wheel, I'd just copy their design.
Here are some of the important fields from sysschedules
freq_type
How frequently a job runs for this schedule.
1 = One time only
4 = Daily
8 = Weekly
16 = Monthly
32 = Monthly, relative to freq_interval
64 = Runs when the SQL Server Agent service starts
128 = Runs when the computer is idle
freq_interval
Days that the job is executed. Depends on the value of freq_type. The default value is 0, which indicates that freq_interval is unused.
Value of freq_type Effect on freq_interval
1 (once) freq_interval is unused (0)
4 (daily) Every freq_interval days
8 (weekly) freq_interval is one or more of the following: 1 = Sunday 2 = Monday 4 = Tuesday 8 = Wednesday 16 = Thursday 32 = Friday 64 = Saturday
16 (monthly) On the freq_interval day of the month
32 (monthly, relative) freq_interval is one of the following: 1 = Sunday 2 = Monday 3 = Tuesday 4 = Wednesday 5 = Thursday 6 = Friday 7 = Saturday 8 = Day 9 = Weekday 10 = Weekend day
64 (starts when SQL Server Agent service starts) freq_interval is unused (0)
128 (runs when computer is idle) freq_interval is unused (0)
freq_subday_type
Units for the freq_subday_interval. Can be one of the following values:
Value Description (unit)
1 At the specified time
2 Seconds
4 Minutes
8 Hours
freq_subday_interval
Number of freq_subday_type periods to occur between each execution of the job.
freq_relative_interval
When freq_interval occurs in each month, if freq_interval is 32 (monthly relative). Can be one of the following values:
0 = freq_relative_interval is unused
1 = First
2 = Second
4 = Third
8 = Fourth
16 = Last
freq_recurrence_factor
Number of weeks or months between the scheduled execution of a job. freq_recurrence_factor is used only if freq_type is 8, 16, or 32. If this column contains 0, freq_recurrence_factor is unused.
Well, to store the recurrence rule itself, you could use a cut down version of RFC 5545 (and I really suggest you cut it down heavily). Aside from anything else, that will make it easy to export into other applications should you wish to.
After you've made that decision, for the database side you need to work out whether you want to store each occurrence of the event, or just one record for the repeated event, expanding it as and when you need to. Obviously it's considerably easier to query the database when you've already got everything expanded - but it makes it trickier to maintain.
Unless you fancy writing some pretty complex SQL which may be hard to test (and you'll want a lot of unit tests for all kinds of corner cases) I would suggest that you make the database itself relatively "dumb" and write most of the business logic in a language like Java or C# - either of which may be embeddable within stored procedures depending on your database, of course.
Another thing you need to ask yourself is whether you need to cope with exceptions to events - one event in a series changing time/location etc.
I have some experience with calendaring (I've spent most of the last year working on the calendar bit of Google Sync via ActiveSync) and I should warn you that things get complicated really quickly. Anything you can deem "out of scope" is a blessing. In particular, do you need to work in multiple time zones?
Oh, and finally - be very, very careful when you're doing actual arithmetic with calendar operations. If you're going to use Java, please use Joda Time rather than the built-in Calendar/Date classes. They'll help you a lot.
The accepted answer here is too convoluted. For example, if an event occurs every 5 days, the 5 is stored in freq_interval, but if it occurs every 5 weeks, the 5 is stored in freq_recurrence. The biggest problem is that freq_interval means three different things depending on the value of freq_type (number of days between occurrences for daily recurrence, day of the month for monthly recurrence, or days of the week for weekly or monthly-relative). Also, the 1,2,4,8... type sequence is used when it is unnecessary and less than helpful. For example, freq_relative_interval can only be "one of" the possible values. This lines up with a drop-down box or radio button type input, not a checkbox type input where multiple choices can be selected. For coding, and for human readability, this sequence gets in the way and just using 1,2,3,4... is simpler, more efficient, more appropriate. Finally, most calendar applications don't need subday intervals (events occurring multiple times in a day - every so many seconds, minutes, or hours).
But, having said this, that answer did help me refine my thoughts on how I am doing this. After mix and matching it with other articles and going from what I see in the Outlook calendar interface and a few other sources, I come up with this:
recurs
0=no recurrence
1=daily
2=weekly
3=monthly
recurs_interval
this is how many of the periods between recurrences. If the event recurs every 5 days, this will have a 5 and recurs will have 1. If the event recurs every 2 weeks, this will have a 2 and recurs will have a 2.
recurs_day
If the user selected monthly type recurrence, on a given day of the month (ex: 10th or the 14th). This has that date. The value is 0 if the user did not select monthly or specific day of month recurrence. The value is 1 to 31 otherwise.
recurs_ordinal
if the user selected a monthly type recurrence, but an ordinal type of day (ex: first monday, second thursday, last friday). This will have that ordinal number. The value is 0 if the user did not select this type of recurrence.
1=first
2=second
3=third
4=fourth
5=last
recurs_weekdays
for weekly and monthly-ordinal recurrence this stores the weekdays where the recurrence happens.
1=Sunday
2=Monday
4=Tuesday
8=Wednesday
16=Thursday
32=Friday
64=Saturday
So, examples:
So, every 4 weeks on Saturday and Sunday would be
recurs = 2 ==> weekly recurrence
recurs_interval = 4 ==> every 4 weeks
recurs_weekdays = 65 ==> (Saturday=64 + Sunday=1)
recurs_day and recurs_ordinal = 0 ==> not used
Similarly, Every 6 months on the first Friday of the month would be
recurs = 3 ==> monthly recurrence
recurs_interval = 6 ==> every 6 months
recurs_ordinal = 1 ==> on the first occurrence
recurs_weekdays = 32 ==> of Friday
None of this business of having a field that means three entirely different things depending on the value of another field.
On the user interface side of things, I let the user specify a date, start time, end time. They can then specify if they want a type of recurrence other than none. If so, the app expands the relevant section of the web-page to give the user the options required for the stuff above, looking a lot like the Outlook options, except there is no "every weekday" under daily recurrence (that is redundant with weekly recurrence on every mon-fri), and there is no yearly recurrence. If there is recurrence then I also require the user to specify an end-date that is within one year of today (the users want it that way, and it simplifies my code) - I don't do unending recurrence or "end after ## occurrences."
I store these fields with the user selections in my event table, and expand that out in a schedule table which has all occurrences. This facilitates collision detection (I am actually doing a facility reservation application) and editing of individual occurrences or refactoring of future occurrences.
My users are all in CST, and I thank the good Lord for that. It is a helpful simplification for now, and if in the future the user base is going to expand beyond that, then I can figure out how to deal with it then, as a well separated task.
UPDATE
Since I first wrote this, I did add daily occurrence with "Every weekday". Our users had a bit of a hard time with thinking that you could use Weekly recurrence for events happening from Thursday one week to Tuesday the next week and only on weekdays. It was more intuitive for them to have this, even if there was already another way that they could do it.
I have been thinking about this too, although have not implemented it but these are my thoughts for a simple solution.
When setting up an event thats recurring, have the user specify the "end date" and create individual events for each one (based on the recurring options). Because its a recurring event, set a unique "recurring ID" for each of these. This ID will then be used to mark an event as recurring and if you change a future event, you can prompt the user to apply this to the rest of the future events by deleting and recreating the recurring events with a new "recurring ID" which will also differentiate this recurring event from the previously ones that have changed.
Hope this makes sense and would like any comments.
I would record recurring events as two separate things in the database. First of all, in an events table, record each and every occurence of the event. Secondly, have recurrences table in which you record the details that you ask for to set up the recurring event. Start date, periodicity, number of occurences, etc.
Then you might think of tying it all together by putting the PK of recurrences into each of the event records as an FK. But a better design would be to normalise the event table into two tables, one which is just the barebones of an event, and one which has the details, which could now be referring to multiple events. That way every event record, recurring or not, has an FK to the PK of the eventdetails table. Then in eventdetails, record the PK of recurrences somewhere along with agenda, invitees, etc. The recurrence record does not drive anything. For instance, if you want a list of all recurring events, you look through eventdetails for all events with a non-null FK to recurrences.
You'll need to be careful to synchronise all of these things, so that you insert or delete events when the recurrence data changes.
"Aside from anything else"
does this include "the very requirements" ?
"that will make it easy to export into other applications should you wish to."
Do the stated requirements include "it must be easy to export the calendars to other applications" ? My impression was that the problem consisted solely of building the FIRST application.
that said, my own response :
You need to limit yourself/your user on the types of "recurrency" your sytem will be able to support. And "All of the above" or "No Limitations" will not be a valid answer if you/your user want(s) to end up with a usable application.
We have the convention of versioning our builds as [major].[minor].[micro].[revision], e.g. 2.1.2.33546.
Our build-script automatically updates an AssemblyInfo.cs file containing
[assembly: AssemblyVersion("x.y.z.w")]
in order to embed the version-number in the assembly.
But our Subversion-repository just reached revision #65535, which broke our build.
It turns out that each number in the version-number has a maximum value of 65534 (probably due to a Windows-restriction).
Have you encountered this problem? Any good solutions/workarounds?
We like the scheme of embedding the revision-number and we obviously can't just reset our Subversion-server :-)
A bit more Background information:
Why are build numbers limited to 65535?
As this is unlikely to get changed, your options are:
Take the Revision Modulo 65535, which means you are back to 1
Use the Micro-Field in your version number to split the version number by dividing the revision by 1000. That means your version could be 1.0.65.535
Do not store the SVN Revision in the AssemblyVersion, but instead in the AssemblyInformationalVersion. That way your Application can still access it for display purposes, although you can't use Windows Explorer anymore to quickly check the SVN Revision
Do not store the SVN Revision in the AssemblyVersion, but instead in the AssemblyProduct or AssemblyDescription fields. Again, that way your Application can still access it, but also Explorer will now show it in the Property Sheet.
One option might be to just use the [AssemblyFileVersion]; this still raises a warning, but it'll build, at least:
[assembly: AssemblyFileVersion("1.0.0.80000")]
We decided to use the same convention, and due to the limitations of Windows version numbers we chose to drop the "micro" part of our version numbers in order to preserve the revision number. Our version numbers are now [major].[minor].[revision / 10000].[revision % 10000], so the assemblies built from revision 65535 have the version 2.01.6.5535.
According to MSDN, the components of the AssemblyVersionAttribute version number are limited to UInt16.MaxValue - 1 by the assembly meta data, i.e. you can't store larger numbers in an assembly file. The file version, as Marc Gravell suggests, might be enough for you, depending on who will read your version number.
This answer is for people, who use the Azure Build Pipeline, want to insert the BuildId value as last number of the assembly version and have a problem with a too large value of the BuildId. (> 65535)
My solution is to use the last 4 or 5 digits of the BuildId, which are injected into the file AssemblyInfo.cs.
I don't use the modulo operation, because than the version number would look totally different from the BuildId (after reaching the limit). Instead in my solution the "shorted" version looks similar to the BuildId.
Examples:
The AssemblyVersion is 1.0.0.0 and the BuildId is 333. --> The new AssemblyVersion becomes 1.0.0.333. (Small number, no problem.)
The AssemblyVersion is 1.0.0.0 and the BuildId is 55555. --> The new AssemblyVersion becomes 1.0.0.55555. (Still in range.)
The AssemblyVersion is 1.0.0.0 and the BuildId is 66666. --> The new AssemblyVersion becomes 1.0.0.6666. (Uses last 4 digits. More isn't possible.)
The AssemblyVersion is 1.0.0.0 and the BuildId is 111111. --> The new AssemblyVersion becomes 1.0.0.11111. (Uses last 5 digits.)
Easy usage by following steps
Step 1: Define the variable shortBuildId in your pipeline by this snippet.
variables:
- name: shortBuildId # note: The last 4 or 5 digits of the BuildId, because for the assembly version number the maximum value is 65535
value: '[not set]' # will be set by powershell script
Alternatively you could define it like this. It depends on the style how you did define your already existing variables.
variables:
shortBuildId: '[not set]'
Step 2: Insert these tasks above the existing tasks.
The first task creates the short BuildId and saves it to variable shortBuildId.
The second task updates the 4th version field in the file AssemblyInfo.cs. So the short buildId is injected into both, the AssemblyVersion and the AssemblyFileVersion.
Note: In this file you need an assembly version with 4 numbers (e.g. 1.0.0.0). If you have only 3 numbers (e.g. 1.0.0) it will not work.
- task: PowerShell#2
displayName: Define short build ID
# If allowed, use the last 5 digits. If they are larger than 65000, use the last 4 digits. Leading zeros are removed.
# This is needed, because the full build ID can't be used as number for the assembly version.
inputs:
targetType: 'inline'
script: |
$shortId = $env:BUILD_BUILDID
$shortId = $shortId % 100000
if ($shortId -gt 65000) { $shortId = $shortId % 10000 }
Write-Host "Build ID: $env:BUILD_BUILDID --> $shortId"
Write-Host "##vso[task.setvariable variable=shortBuildId]$shortId"
showWarnings: true
- task: RegexReplace#3
displayName: Insert shortBuildId into AssemblyInfo:
InputSearchPattern: 'myProjectDirectory\Properties\AssemblyInfo.cs'
FindRegex: '(\[assembly: (AssemblyVersion|AssemblyFileVersion)\("\d+\.\d+\.[0-9*]+)\.[0-9*]+("\)\])'
ReplaceRegex: '$1.$(shortBuildId)$3'
UseUTF8: true
UseRAW: true
Step 3: Adjust the path in the second task related to your project.
Edit the value of InputSearchPattern.
If you want to insert the shortBuildId into all projects of your solution, just write InputSearchPattern: '**\AssemblyInfo.cs'
Credit
Thanks to Dr. Edmund Weitz for his great tool The Regex Coach, which is free to use.
I'd like to propose by way of answer the following scheme for anyone using semver/gitflow:
AssemblyVersionAttribute
SemVer/Gitflow
Major Version
Major
Minor Version
Minor
Build Number
Patch
Revision
Gitflow ID
Where "Gitflow ID" is a digit followed by 0000 - 9999, per the following:
Gitflow ID
Branch
00000 - 09999
Release (alpha)
10000 - 19999
Release (beta)
20000 - 29999
Release (RC)
30000 - 65535
Development
The intuition behind 00000 - 29999 is that these numbers represent something of a logical negative pre-release number, 30000 represents logical zero, and 30001 - 65535 represent logical positive. More formally, this is a kind of 10's complement representation, with offset K = 30000.
So for example:
Topic branch feature/A starts at 0.0.0.30000
Simultaneously, topic branch feature/B also starts at 0.0.0.30000
feature/B merges to dev at 0.0.0.31000 while feature/A is at 0.0.0.30021
feature/A merges updates from dev at 0.0.0.31001
feature/A merges to dev at 0.0.0.32000
v1.0-alpha.1 release starts from dev at 1.0.0.00001
v1.0-rc.3 at 1.0.0.20002
Finally v1.0 released to Master at 1.0.0.30000
Hotfix v1.0.1 applied at 1.0.1.30000
Meanwhile v1.1 development continuing at 1.0.1.30002
The above suggests that the development range 30000-65535 could be further subdivided for topic branches, i.e. DDTTT, with DD ranging from 30 to 65 (max 65 - 30 + 1 = 36 dev PRs until release). Alternatively, the whole range could be used for development and topic branches without distinction; in this case, merging from dev to topic would have topic be dev + 1 and vice-versa. Either case allows there to be multiple identical version numbers at the topic branch level, but only a single version number for any dev commit. The DDTTT arrangement makes it clearer which version number represents a dev commit (e.g. 57000) at the expense of limiting the number of dev commits in a release. However, assuming a frequent enough release cadence, this should not be a problem. At any rate, production releases are clearly seen as having gitflow IDs of 30000.
In this code I am debugging, I have this code snipit:
ddlExpYear.SelectedItem.Value.Substring(2).PadLeft(2, '0');
What does this return? I really can't run this too much as it is part of a live credit card application. The DropDownList as you could imagine from the name contains the 4-digit year.
UPDATE: Thanks everyone. I don't do a lot of .NET development so setting up a quick test isn't as quick for me.
It takes the last two digits of the year and pads the left side with zeroes to a maximum of 2 characters. Looks like a "just in case" for expiration years ending in 08, 07, etc., making sure that the leading zero is present.
This prints "98" to the console.
class Program
{
static void Main(string[] args)
{
Console.Write("1998".Substring(2).PadLeft(2, '0'));
Console.Read();
}
}
Of course you can run this. You just can't run it in the application you're debugging. To find out what it's doing, and not just what it looks like it's doing, make a new web application, put in a DropDownList, put a few static years in it, and then put in the code you've mentioned and see what it does. Then you'll know for certain.
something stupid. It's getting the value of the selected item and taking the everything after the first two characters. If that is only one character, then it adds a '0' to the beginning of it, and if it is zero characters, the it returns '00'. The reason I say this is stupid is because if you need the value to be two characters long, why not just set it like that to begin with when you are creating the drop down list?
It looks like it's grabbing the substring from the 3rd character (if 0 based) to the end, then if the substring has a length less than 2 it's making the length equal to 2 by adding 0 to the left side.
PadLeft ensures that you receive at least two characters from the input, padding the input (on the left side) with the appropriate character. So input, in this case, might be 12. You get "12" back. Or input might be 9, in which case, you get "09" back.
This is an example of complex chaining (see "Is there any benefit in Chaining" post) gone awry, and making code appear overly complex.
The substring returns the value with the first two characters skipped, the padleft pads the result with leading zeros:
string s = "2014";
MessageBox.Show(s.Substring(2).PadLeft(2, 'x')); //14
string s2 = "14";
MessageBox.Show(s2.Substring(2).PadLeft(2, 'x')); //xx
My guess is the code is trying to convert the year to a 2 digit value.
The PadLeft only does something if the user enters a year that is either 2 or 3 digits long.
With a 1-digit year, you get an exception (Subsring errs).
With a 2-digit year (07, 08, etc), it will return 00. I would say this is an error.
With a 3-digit year (207, 208), which the author may have assumed to be typos, it would return the last digit padded with a zero -- 207 -> 07; 208 -> 08.
As long as the user must choose a year and isn't allowed to enter a year, the PadLeft is unnecessary -- the Substring(2) does exactly what you need given a 4-digit year.
This code seems to be trying to grab a 2 digit year from a four digit year (ddlexpyear is the hint)
It takes strings and returns strings, so I will eschew the string delimiters:
1998 -> 98
2000 -> 00
2001 -> 01
2012 -> 12
Problem is that it doesn't do a good job. In these cases, the padding doesn't actually help. Removing the pad code does not affect the cases it gets correct.
So the code works (with or without the pad) for 4 digit years, what does it do for strings of other lengths?
null: exception
0: exception
1: exception
2: always returns "00". e.g. the year 49 (when the Jews were expulsed from rome) becomes "00". This is bad.
3: saves the last digit, and puts a "0" in front of it. Correct in 10% of cases (when the second digit is actually a zero, like 304, or 908), but quite wrong in the remainder (like 915, 423, and 110)
5: just saves the 3rd and 4th digits, which is also wrong, "10549" should probably be "49" but is instead "54".
as you can expect the problem continues in higher digits.
OK so it's taking the value from the drop down, ABCD
Then it takes the substring from position 2, CD
And then it err, left pads it with 2 zeros if it needs too, CD
Or, if you've just ended X, then it would substring to X and pad to OX
It's taking the last two digits of the year, then pad to the left with a "0".
So 2010 would be 10, 2009 would be 09.
Not sure why the developer didn't just set the value on the dropdown to the last two digits, or why you would need to left pad it (unless you were dealing with years 0-9 AD).